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FOREWORD 


The present publication represents the final Report of the 
Scottish Committee, or Delegation, in the International Examina¬ 
tion Inquiry initiated by the Carnegie Corporation of America, 
the Carnegie Foundation and the International Institute ol 
Teachers College, Columbia University, and financed jointly by 
the Carnegie Corporation and the Carnegie Foundation. It 
also represents the completion of the tasks undertaken in 1931 at 
the Eastbourne Conference by the Scottish Delegation. This, the 
fourth of our major investigations, had not defined itself very 
clearly until just before the second Conference, held at Folkestone 
in 1935, where the plans were submitted for criticism and for 
approval. The third Conference, at Dinard in 1938, saw the 
investigation so far advanced that Professor McClelland was 
able to give a detailed account of what had been completed and 
what still remained to be done. Now, four years later, he 
is able to present to the Committee and to the public the 
full results. It will, I think, be generally agreed that this has 
been the most important and the most onerous of all the investiga¬ 
tions sponsored by the Committee; it consequently seems incum¬ 
bent on me as Convener to express our indebtedness to Professor 
McClelland, who has throughout borne on his own shoulders 
practically the whole burden. The measure of that indebtedness 
can be assessed by the reader of this work. 

At the Eastbourne Conference the Chairman, Dr Paul Monroe, 
stressed the sociological implications of examinations, and pointed 
out that these implications constitute the most general aspect of 
the whole series of investigations undertaken by the various 
national delegations. The enormous sociological significance of the 
present investigation has been indicated by Professor McClelland, 
and any elaboration is superfluous. It is worth noting, however, 
that the Mental Survey investigations of 1932 and 1935-37 might 
be said to represent the first stage in a great educational and 
sociological study, of which the present investigation represents 

the second stage, and that a third stage remains to be undertaken_ 

a mental and educational survey of the population of our secondary 
and technical schools, our universities and central institutions, as 
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well as our adult education classes. This third stage has already 
been envisaged by the Scottish Council for Research in Education, 
but financial considerations preclude for the time being the 
possibility of undertaking an investigation so complex, so difficult 
and so costly. 

Although the International Examination Inquiry as initiated 
and financed by the International Institute of Teachers College 
has now terminated, at least so far as the Scottish Delegation is 
concerned, we are still faced with this great task, and with the 
study of other problems concerned with examinations. The hope 
may be expressed that, when the present difficult days are past, 
it will again be possible to secure international collaboration in 
this work. At the closing session of the Dinard Conference in 
1938, on the motion of the Convener of the Scottish Delegation, 
it was unanimously agreed that an international committee for 
the study of examinations be appointed, and that it consist in 
the first instance of the chairmen of the existing committees. 
Subsequently Professor Godfrey H. Thomson consented to act as 
secretary to such a committee. Unfortunately the unsettled 
state of the Continent, and then the War, made it impossible 
to carry the organisation of the committee beyond the skeleton 
stage. When peace returns, it may not be too much to expect 
that this committee will begin to function, and continue the 
work so successfully prosecuted for seven years under the aegis 
of the International Institute, and the inspiring leadership of 
Dr Paul Monroe. 

It only remains to express once more in the name of the Scottish 
International Examinations Inquiry Committee and the Scottish 
Council for Research in Education their sense of the deep debt 
they owe—and Scottish education owes—to the International 
Institute of Teachers College for the continued encouragement 
they have received, and for the very substantial financial assistance 
without which the investigation could not have been undertaken 
and published. 

JAMES DREVER, 

Convener. 



PREFACE 


This work is a scientific study of certain problems of selection 
and guidance that arise at the first big sifting of the material in 
an educational system; and, as such, it is not directly concerned 
with many of the deeper issues of educational and examination 
policy. We set out to find answers to a number of definite 
questions in which administrators are interested, but we offer no 
opinion as to whether, in a sound educational system, the questions 
would arise in these particular forms, or would, indeed, arise at all. 

Our thanks are due to the Convener and Members of the 
Dundee Education Committee not merely for their ready and 
generous response to our application for facilities to conduct the 
experiment, but also for the encouraging interest which they 
have taken in its progress. To their Director of Education, 
Mr John R. Cameron, M.A., we have been indebted for the 
preparation of the examination papers and for constant guidance 
and help in all our difficulties. Dr A. E. Kidd, M.B., C.M., 
D.P.H., former Chief Medical Officer for Schools, and his staff 
undertook the heavy task of furnishing us with health gradings and 
medical reports for the complete group of over 3,000 pupils; and 
Dr Kidd also prepared the classification of schools on the basis 01 
social class. We have also to record our appreciation of the co¬ 
operation of the head teachers and staffs of the City schools, to 
whom we are deeply grateful for the friendly and helpful way in 
which they met our heavy demands upon their time and patience. 

Lengthy as the Report is, it is only to those with actual 
experience of similar research that it will convey a full realisation 
of the labour involved. The determination of a single correlation 
coefficient, when the numbers are large, is a long and laborious 
operation; and, to ensure accuracy, all our calculations were done 
twice, by different teams. With this duplication the number 
of direct correlation calculations in the main Inquiry was over 
3,000: the testing programme involved the correction of over 
20,000 scripts. 

To embark upon such an undertaking without a large full-time 
staff implied a belief, which events have proved to be well-founded, 
that we could safely count upon the help of our students and of 
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our colleagues in the City schools. The Inquiry was, in fact, an 
adventure in large scale co-operative research carried out by 
voluntary part-time assistants; and a brief description of the 
organisation may be of interest to other investigators. 

My two partners in the general conduct of the Inquiry were 
Miss Margaret Young, M.A., B.Ed., Lecturer in Experimental 
Education in Dundee Training College, and Douglas M. McIntosh, 
M.A., B.Sc., B.Ed., Ph.D,, Assistant Director of Education for 
the County of Fife. Miss Young’s special responsibilities included 
charge of the testing, preparation of circulars of instructions, 
training teams of testers and correctors, etc. She is the writer 
of Chapter I, in which she gives an account of these matters. 
Dr McIntosh, who has written Chapter II, undertook such tasks 
as the standardisation of the tests, organisation of the correlation 
calculations, training teams of computers, and so on. Yet neither 
a list of special duties nor the occasional references in the Report 
can give an adequate appreciation of what the Inquiry owes to 
these two principal assistants. Each envisaged the research as a 
whole and helped with practically every aspect of it. 

So also did Miss Muriel Mitchell, M.A., who, before her 
appointment to the administrative staff of the College, had charge 
of the rapidly mounting mass of documents in the room set aside 
for the Inquiry. Since 1936 she has given voluntary help of the 
utmost value and has been much in request as a consultant to 
whom student assistants went singly or in groups for advice in 
their statistical difficulties. 

The correction of the tests and the correlation calculations 
were carried out by a veritable army of helpers, over 600 in all, in 
whose ranks were members of the College staff, head teachers 
and teachers of the City schools, and students of the Training 
College. Most of them had to be trained for the work, and this 
training was given in the first place by Miss Young and Dr 
McIntosh. Later, certain of the more skilled workers acted as 
group leaders and trained new teams. Two of these group 
leaders, Alex. S. Robertson, M.A., and James C. Kidd, M.A., B.A., 
gave particularly valuable help with the more difficult calculations 
and many special problems. 

The teams of correctors and computers fell roughly into two 
types, the ‘permanent’ and the ‘occasional.* Into the former 
came a most faithful group who worked from 6 to 9 p.m. on one 
or two evenings a week for several sessions. In the latter we had 
groups of students who worked in forenoons during vacations; 
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others who worked in free periods during the College session; 
others who worked individually at home, coming up to the 
evening meetings periodically to discuss difficulties. 

Finally, about fifty graduate students undertook studies of 
special aspects. Each year, after a talk on the aims and progress 
of the Inquiry, a list of problems was submitted from which 
interested students could make their choice. There was no lack 
of volunteers for these investigations, many of the results of which 
will be found in the Report. 

It may not be out of place to say a word as to the part played 
by the Inquiry in the life of the Training College. The students* 
help was invaluable, and our gratitude to them is not diminished 
by the belief that the benefits were mutual. For many of them 
simple statistics forms part of the course of training, and the 
calculations which they carried out in connection with the 
Inquiry merely replaced class-room exercises which would have 
had less reality and interest. Moreover, while all types of student 
participated, they were not asked to work in a blind mechanical 
manner. Before setting to work they understood the plan of 
the research and the way in which their particular calculation 
fitted into the whole. Whenever possible we let them know the 
results which their group had obtained, and discussed their 
significance. In short, the Inquiry was a central research interest 
for the whole College for six years. Its general headquarters, the 
Inquiry room, was a meeting-place for students from all sections 
who were co-operating in the work. It had a real research 
atmosphere, and no one could associate with the students in it 
without a feeling that the project was an enriching influence in the 
life of the College. 

Of the many colleagues who helped us we are specially indebted 
to Mr B. Babington Smith, M.A., of the Psychology Department 
of the University of St Andrews, with whom we had many 
fruitful discussions on statistical problems, and to the following 
members of the staff of the Dundee Training College: Mr 
Andrew Nairn, Principal Lecturer in Art, who prepared the 
diagrams; Mr Robert B. Martin, Chief Clerk, who acted as 
Treasurer; Miss Dorothy Mess, and other members of the 
administrative staff whose help included the typing and duplicating 
of circulars, etc.; and Mr George Caithness, janitor, who helped 
us in ways too numerous to specify. 

Throughout the Inquiry, from the original formulation of the 
plan to the final preparation of the Report, we had the benefit of 
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the'expert advice of the Research Council’s Director, Dr R. R. 
Rusl^. The extent of our indebtedness to him will be best 
appreciated by those who have carried out researches for the 
Council. 

Finally, a reference may be made to the Report itself. It is 
primarily a research report; and the ways of educational research 
have become very technical: yet we have tried to present it in 
such a way that the results will riot be rendered inaccessible to 
the practical administrator through being expressed in the un¬ 
intelligible mathematical jargon that would be more pleasing to the 
expert. We have kept the mathematics as far in the background 
as possible, and we believe that the lay reader, whose statistical 
attainments are limited to a knowledge of the meaning of the 
terms ‘standard deviation* and ‘correlation coefficient,* will be 
able to follow the text.' He should simply ignore the footnotes. 
These are for those who are interested in the techniques; but to 
avoid possible disappointment, it should be mentioned that most 
of our methods and formulae had to be devised ad hoCy and that 
the references often give merely the starting-point from which it 
will be possible to retrace our mathematical steps. 

We have tried to throw a little light on some of the dark places 
of the selection problem: but by the time the reader has finished 
the Report he will realise that the torch has been a heavy one to 
carry—and we now willingly hand it on to others. 
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CHAPTER I 

PLAN AND ORGANISATION OF THE INQUIRY 

The Qualifying Stage in Scottish Education 

What is known as the Qualifying stage in Scottish education is 
that at which a pupil is transferred from the uniform primary 
course to differentiated post-primary courses. The normal age 
of transfer is 12 years. 

Until December 1921 all pupils who had completed a primary 
course had to undergo a Qualifying examination, conducted by 
H.M. Inspectors, the function of which was to ascertain whether 
the pupil had satisfactorily completed the primary stage and was 
fit to enter a post-primary course. When this examination was 
abolished, the Scottish Education Department indicated, in 
Circular No. 44^ that the Qualifying stage ‘ must continue to mark 
something of an epoch in the school career of every pupil’; and 
Education Authorities were advised to substitute for the Depart¬ 
ment’s Examination ‘some arrangement under which those who 
have taught and those who are to teach particular individuals 
shall combine in an endeavour to estimate the potentialities of the 
material to be handled.’ 

Since 1922 there has been a considerable amount of experiment 
with different methods of complying with this injunction. Each 
Education Committee has evolved its own scheme of promotion 
at the Qualifying stage, and these schemes are modified from time 
to time in the light of experience gained. They range from 
smple systems where only one measure of attainment is used, to 
elaborate systems involving the use of standardised mental and 
scholastic tests, ordinary examinations and teachers’ estimates. 

,made of the results of the examinations at 
the Qualifying stage vary from Area to Area. 


The Object of the Inquiry 

The aim of ^3 Inquiry is to study the problems of selection 
and guidance that arise at the Qualifying stage, and to find the 
system of assessment of ability and attainment which will give the 
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most reliable forecast of the type of post-primary course for which 
each pupil is fitted. In particular the object is to find answers 
to the following questions: 

What are the relative predictive values of (a) ordinary examina¬ 
tions, (b) teachers’ estimates, (f) group intelligence tests and 

(d) standardised scholastic tests? 

What are the best combinations of the above measures in 

selecting pupils for secondary courses? 

The General Plan of the Inquiry 

Satisfactory answers to such questions can be obtained only 
by means of a large-scale follow-up investigation. We therefore 
first tested the complete Qualifying group in a Scottish City by 
all the methods of assessment at present in use, obtaining additional 
data on special rating cards. The progress of these pupils in the 
various post-primary courses was then followed for three years. 

We proposed to continue the follow-up of the senior secondary 
pupils for five years, but this had to be abandoned because of war 

conditions. 

THE QUALIFYING GROUP 

The Qualifying group consisted of aU pupils P™', 

motion to post-primary courses in Session ^ 93 $-^ Jecembei 
number was 3,229, of whom i 130 were presented ° ^ecembe 

iq« and 2,099 in June 1936. In view of the two dates 01 

p?Lentation,Tplicate tests and examinations had to be prepared. 
The age distributions of the two groups were as foUows. 


Age 

H-'S 

13-14 

12-13 

11-12 

lO-II 


December 1935 

N 

Per cent. 

3 

■3 

46 

4 -* 

578 

511 

501 

44'3 

2 

•2 

1,130 



June 1936 
N Per cent 


9 

126 

1,018 

9+0 

6 

2,099 


•4 

6‘0 

48-5 

44.8 

•3 


THE AGE-GROUPS 

The aee-groups were used for the standardisation of the in¬ 
ternee® aid scholastic tests. They consisted of aU pupds m 
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the City of ages from 11 years 6 months to 12 years 5 months on 
the dates of the two examinations. The distributions of these 
age-groups among the classes were as foUows: 



December 1935 

June 

1936 

Class 

N 

Per cent. 

N 

Per cent. 

Primary 

III . 

48 

1-6 

45 

t -5 

IV . 

249 

81 

348 

11-9 

V (Qualifying class) . 

1,860 

603 

1,940 

667 

Secondary 

First year . 

906 

294 

576 

198 

Second year 

*9 

•6 

2 

•1 


3,082 


2,911 



THE QUALIFYING DATA 

Standardised Tests 

The following Moray House Tests prepared by Professor 
Godfrey H. Thomson were used: ^ 


Test Duration in minutes 


Subject 

December 

1935 

June 

1936 

Practice Test 

Total 

Intelligence 

. MHT 20 

MHT 21 

10 45 

55 

English 

. MHE 7 

MHE 8 

40 

40 

Arithmetic 

, MHA 7 

MHA 8 

30 

30 


Qualifying Examinations 


Subject 

Description 

Duration in minutes 

English 

. Dictation and Spelling 

. 30 approximately 

Composition 

General English 

• 35 


(<a) Interpretation of passage 


{b) Simple questions on 

English usage 6o 

Arithmetic . 

. Mental 

• . . 10 


Mechanical and problems 

. • . 60 


* Particulars of the Moray House Tests may be found in Wbat are Moray 
UouieJesuT London: University of London Press, Ltd., 1940. 
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The papers, which were based upon those set in recent years to 
Qualifying pupils in various Scottish counties, were prepared by 
the Director of Education for the City. Copies of the December 
papers are given in Appendix A, 

^eacber^ Estimates of Attainment in the Primary School 
The head masters and teachers were in the habit of estimating 
the attainment of each pupil at the Qualifying stage, and these 
estimates were obtained from the schedules sent to the Director 
of Education. Marks were on a percentage basis in English and 
Arithmetic, while a qualitative grading was given in practical 
subjects such as Handwork and Drawing. 

Bursary Examinations 

The City Education Committee allowed the results of the 
bursary examinations in English and Arithmetic set to applicants 
for bursaries for senior secondary courses to be used. 

Rating Cards 

A rating card was devised for the purpose of securing qualitative 
estimates of each child’s health, ability, attainment and personality. 
The primary head masters were responsible for completing this 
card, guided by a carefully prepared memorandum of instructions. 
The grading for health was supplied by the Chief Medical Officer 
for Schools. This rating card is shown in Appendix B. 

ADMINISTRATION OF THE TESTS AND EXAMINATIONS 

Compilation of a Card Index 

Particulars regarding each pupil were supplied by head masters 
on special cards. These cards provided an estimate of numbers 
in each school and particulars for the tabulation sheets which had 
to be prepared before the tests and examinations were corrected. 

Arrangements in Schools 

Head masters co-operated in securing the best possible testing 
conditions. Pupils were generally tested in groups of 25 to 30, 
seated at single desb. Circulars were prepared to ensure uni¬ 
formity, and in deciding matters relating to internal school 
arrangements the assistance of the Director of Education was 
invaluable. The following circulars to head masters were 

issued: 

Circular No. 1 . Mental and Scholastic ^esU. This gave 
instructions regarding time-table, arrangement of pupils in 
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groups, names of examiners and supervisors, material required, 
return of material. 

Circular No. 11 . Mental and Scholastic Tests. This brief 
memorandum confirmed the times stated in Circular No. I 
and gave full details of the duration of the tests. 

Circulars of a similar type were issued in connection with the 
examinations. 

Time-table of Tests and Examinatiotis 

The intelligence and scholastic tests were given on three con¬ 
secutive days. The examinations were given on two consecutive 
days about a fortnight after the last test. 

Distribution and Collection of Material 

Great care was exercised in the distribution and collection of 
standardised test and examination papers. Sealed packets were 
delivered to each head master and handed by him to an accredited 
supervisor before each test period. The standardised tests had 
not previously been used in the City. All used and unused tests, 
all examination question papers and examination scripts, were 
returned to the Dundee Training College at the close of each 
day’s testing. 

Teams of Testers and Supervisors 

The standardised tests were administered by teams of teachers 
and students trained in this technique. The same teams con¬ 
ducted the Qualifying examinations under carefully controlled 
conditions. Meetings were arranged with teachers and students 
at which details of procedure were discussed, and an accredited 
supervisor was appointed to each school to make contact with 
the head master. The following circulars were distributed to the 
testers and supervisors: 

Memorandum for Testers No. 1 . Mental and Scholastic Tests. 
DetaUs were given of dates and duration of test periods, and 
instructions as to test procedure were carefully stated. 

Mmorandum for Testers No. II. Qualifying Examinations. 
Details were given as to distribution of examination material, 
heading of answer sheets, and administration of examinations! 

Memoranda for Supervisors of Tests and Examinations. These 
memoranda gave instructions for the distribution and coUec- 
tion of test material, examination material and absentee slips. 
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The care exercised in the conduct of the ordinary type of 
examinations was occasioned by the belief that such examinations 
may be at a disadvantage relative to standardised tests if they 
are given under ordinary school conditions while the tests are 
administered under rigorous research conditions. 


Correction and tabulation of Results 

(a) Standardised tests. The efficient administration of the 
tests and examinations was due to the loyal team work of head 
masters and teachers of primary and secondary schools assisted by 
Training College students. Their continued co-operation made 
light work of the formidable task of correcting the 10,000 standard¬ 
ised tests. The correction, checking, calculation of quotients and 
tabulation of results were carried out in the Training College by 
teams of volunteers under supervision. 

(b) Qualifying examination scripts. The Qualifying examina¬ 
tion scripts were marked by a paid examiner who has for many 
years corrected the Qualifying examination papers in a Scottish 
Education Area. 


THE FOLLOW-UP 

Tbe System of Promotion to Post-Primary Education 
At the time of the testing in Session 1935-1936 promotion to 
post-primary education in the City was based on the estimates 
of the pupils* attainment given by the primary schools. On this 
basis the Education Committee decided which pupils should be 
transferred to special classes for backward children or retained in 
primary schools for later presentation. Pupils regarded as fit 
for promotion to three-year or five-year secondary courses were 
guided by the head teachers of the primary schools as to choice 
of course, but there was no compulsion. 

The only selection for five-year secondary education resulted 
from the fact that, at this time, fees were charged in all the senior 
secondary schools. Pupils desiring a full secondary education, who 
would have been debarred by reason of the expense involved, took 
a special examination. If they attained a certain standard in this 
examination, defined from year to year by the Education Com¬ 
mittee, they were admitted free to a senior secondary school. 

The results of the standardised tests and Qualifying examina¬ 
tions given in this investigation were not used to influence 
promotion. 
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Tracing the Qualifying Group 

The tracing of the 3,229 pupils presented for the Qualifying 
examination in 1935-1936 proved to be an intricate task. The 
courses to which they proceeded are shown below: 


Course 


N 

Per cent. 

Senior secondary courses. 

Junior secondary courses 

• 

461 

H *3 

Boys’ technical .... 

• 

870 

26*9 

Girls’ technical .... 

• 

763 

236 

Commercial ..... 

• 

379 

117 

Special classes for backward pupils . 

Retained in primary schools with a view to later 

3 S 7 

III 

presentation ..... 


342 

10-6 

Others: left City, etc. .... 

Record Booklets 


57 

3.229 

1-8 


The Dundee Post-Primary School Record Booklets were used to 
record the educational history of the pupils concerned in the 
investigation. The original booklets were sent to the Training 
College at the end of each examination period and a duplicate 
record was made of the marks of each pupil. 

Final Follow-up Cards 

Final follow-up cards were devised for the purpose of securing 
a general estimate of the pupil’s progress at the conclusion of the 
first three years of the post-primary course, or when he left school. 
Three types of card were required: for pupils in senior secondary 
schools; for pupils in junior secondary schools; for pupils in 
backward classes. Detailed instructions for completing these 
final follow-up cards were sent to the head masters of the schools 
concerned. A follow-up card is shown in Appendix C. 



CHAPTER II 

STANDARDISATION OF THE TESTS 

The results of intelligence and scholastic tests may be used either 
in the form of scores or of quotients; that is to say, without or 
with an allowance for differences in the ages of the candidates. 
As one of our aims was to find whether the score or the quotient 
should be used in selection, we required a reliable means of trans¬ 
forming scores into quotients, and this involved the standardisation 
of all the tests. 

Reasons for not using Previous Standardisations 

The Moray House Tests which were used had already been 
standardised for an English county. At first it was thought that 
these standardisations might be suitable for the purpose of the 
Inquiry, but there were several objections to this procedure. 

Firstly, there was doubt as to whether the results from an 
English county with a large rural population would be similar to 
those from a Scottish industrial city. Again, the conditions under 
which tests are given in England differ from those under which 
such an experiment is carried out in Scotland. For example, it 
is a general rule in England that these tests are given to deter¬ 
mine which children shall obtain free places in a secondary school, 
and this gives the candidates an incentive to do well. Added to 
this is the fact that the tests have been given for several years in 
the county, with the result that the children have acquired a 
certain familiarity with the form of the questions. 

These difficulties had already been experienced by the Scottish 
Council for Research in Education when it undertook a national 

survey of a complete age-group in 1932.^ 

The comparability of the norms might also be affected by the 
fact that the groups selected for the standardisation of the tests 
in England were at a lower age-level than that of the group tested 
in the Inquiry. 

» The Jmelligence of Scottitb Children. PubUcations of the Scottish Council 
for Research in Education No. V. London: University of London Press, Ltd., 

*933- PP- 72-79* 
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From a previous survey it was found that the average age of the 
Qualifying pupils in the City was 12 years. As the ages of 73 per 
cent, of the pupils lay within the age-group 11 years 6 months to 
12 years 5 months it was decided to standardise the tests on this 
group, and to extend the norms on either side of the age-limits. 

The distributions of the children in the age-groups for December 
and June have already been given on p. 3. 

Method of Standardisation 

The tests were standardised by a method devised by Professor 
Godfrey H. Thomson.‘ 

Comparison of Inquiry and English Results 

The previous standardisations enable us to make some com¬ 
parisons between Inquiry and English children in regard to 
intelligence and scholastic attainment. If Scottish readers find 
the results to be somewhat unexpected, they must remember, in 
addition to the special explanations already given, that the in¬ 
telligence level of a country is not uniform. 


intelligence tests 


The comparison on the basis of MHT 20 is given in fig. i, 
which will be intelligible to those who are not familiar with 
Professor Thomson’s technique of standardisation. The lines 

may be regarded as giving the average scores of pupils of different 
ages at various levels of ability. 

The first point to be observed is that the groups on which the 
standardisations are based are at different age-levels, the English 
group ranging from 10 years 9 months ^ to 11 years 8 months while 
the Inquiry group range from ii years 6 months to 12 years 
5 months. The most outstanding feature of the diagram is the 
superiority of the English group at every level of ability. Another 
teature is the parallelism of the corresponding lines, indicating that 
the age-aUowances at the different levels are in close agreement. 
When translated mto terms of IQ the difference between the two 


Standardisation of Group Tests and the Scatter 

Part H «/ Educational Psychology, vol. ii, 

Part II, June 1932, pp. 92-137. •' 

ia the 6gurt?^ comparison 13 shown 


I* 




Fic. I.—IntelligcDce: Comparison between English and Inquiry 

pupUs (MHT zo). 
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standardisations amounts to something in the region of 7 to 8 
points. 

At first sight it would appear that the English children are 
superior in intelligence to those of the Inquiry group, but several 
important points must be considered before any such conclusion 
can be drawn. These, as already mentioned, include the 
familiarity of the English children with the form of questions 
and the incentive to do well in such tests. 

It is important to note that the process of standardising the 
test on the City age-group makes the mean IQ of this group 100 
and the standard deviation 15. 

ENGLISH TESTS 

Fig. 2 provides a comparison between the results of English 
and Inquiry pupils on one of the English tests. This is much 
more favourable to the Inquiry group. They are superior at the 
95-percentile, about equal at the 50-percentile, and show a greater 
rate of increase of score at all levels. 

ARITHMETIC TESTS 

Fig. 3 gives the comparison for MHA 7. Perhaps the most 
striking feature is the small rate of increase in score for increasing 
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# 

age at the high levels of ability in the Inquiry group. A possible 
explanation is that the clever children reach a uniform standard 
of achievement where variation in score is negligible. This may 
arise from the teacher’s tendency to spend a great deal of time in 
bringing the duller children up to a certain level, thus giving little 
opportunity for the clever child to develop his ability to the full 
extent. With the English pupils the rate of increase at the high 
level of ability is greater than that at the lower levels, indicating 
that the clever children are making greater relative progress than 
the duller children. At all levels the English pupils show a 
performance superior to that of the Inquiry group. 

The comparisons are all in respect of the December group, which 
is slightly poorer in quality than that presented in June. In 
general, however, the results for the June testing are in close 
agreement. 



CHAPTER III 

RELIABILITY OF THE TESTS AND EXAMINATIONS 


Classification for Secondary Education on the Basis 
OF Duplicate Qualifying Examinations 

On the basis of the total marks the examiner was asked to divide 
the pupils into four categories: 

A. Those fitted for a senior secondary course leading to the 

Senior Leaving Certificate. 

B. Those fitted for a junior secondary course leading to the 

Junior Leaving Certificate. 

C. Those fitted for a junior secondary course of a lower standard 

than B. 

D. Those not fitted for promotion to any of the above types of 

course. 


Category 


Number of pupils in category 
in first examination 



126 




>57 


Distribution in second 
examination 



HI 

p 

H 


1 

Id 

0 

A 

25 

B 

36 

C 

21 

Id 

0 


3 

|b 

22 


112 

Id 

20 




fA 

0 

D 

>35 

|b 

C 

0 

28 



Id 

107 


500 


500 


^ To test the reliability of this classification a duplicate examina¬ 
tion was set to the June group and the scripts of a representative 
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sample of 500 were corrected. The sample was selected by 
arranging the cards of the pupils who sat the first examination in 
order of merit and drawing every fourth. This gave nearly 500, 
the number being completed by drawing a few cards at random. 
The classification of pupils in the two examinations is shown 
on p. 13. 

The agreement, particularly in the A and D categories, is much 
closer than one would expect from some of the data recently 
published on the unreliability of ordinary examinations, and it 
suggests that the correction and classification had been most 
carefully performed. 


Reliability Coefficients of the Tests and Examinations 


Another method of estimating reliability is to find the correla¬ 
tion between the two series of scores obtained when duplicate 
tests are applied to the same pupils. This is a reliability coefficient^ 
and Dr Douglas M. McIntosh determined it for the Qualifying 
examination by using the results for the sample of 500. For the 
intelligence and scholastic tests he found the necessary data in 
the fact that nearly 1,400 pupils were common to the age-groups 
tested in December and June. 

If we examine the raw coefficients in the first column of Table I 
perhaps the first point that strites us is that the correlations for 
the ordinary examinations are unexpectedly high. It is also 
surprising to find that the reliability is higher for English than 
for Arithmetic, particularly in the examinations, where the 

English papers included an essay. 

Change in order of marks from one examination or test to 
another may be the result of two causes: variation in the standard 
of performance of the individual pupils due to such factors as 
health, fatigue, suitability of the paper; and variation in the 
standard of marking of individual papers by the examiner. In 
the examinations both causes are operative; in the tests only the 
first, for the marking is done by a standardised system. For this 
reason we should expect the reUabiHty of the tests to be higher 


than that of the examinations. 

ReliabiHty coefficients have to be interpreted with some 
statistical caution, since their size depends upon the length of the 
tests. In a sense it is unfair to compare the reliability coefficient 
of an intelligence test of 55 minutes with that of an examination 
lasting 3i hours. For a fair comparison we should reduce them 
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aU to the same length, say an hour; and when this is done 1 we 
get the coefficients in the right-hand column of Table I. The 
tests now rise to the place which one would expect, and Q falls 
from the top of the list to the bottom. In this column Q is not 
Qe + Qa as in the first column, but an examination with half-hour 
papers in English and Arithmetic. 

TABLE I 


Reliability coefficients (raw and adjusted for length): 

Tests and examinations 


Test or 
examination 

Reliability 

coefficient 

Test or 
examination 

Reliability coefficient if all 
had been of same length 

Q* 

+ •933 

E 

+ ■953 

E 

+ •931 

A 

+ •934 

Qe 

+ •920 

I 

+ •921 

I 

+ •898 

Qe 

+ •847 

A 

+ •877 

Qa 

+ •845 

Qa 

+ •864 

(Q 

+ •811) 


Qe, Qa, Q: N = 5oo I, E, A: N = i,350 


• An explanation of the symbols used in the report will be found in the 
Glossary of Symbols in Appendix D. 


Standard Errors of Measurement 

A reliability coefficient depends also on the range of ability in 
the group for which it is calculated, and this factor must now be 
considered. We could correct the above coefficients for range, 
but for various reasons we use a different parameter, the standard 
error of measurementy which is small with a reliable test and large 
with an unreliable one. It has the advantage of being independent 
of the range. We can also calculate what it would be if the tests 
were all of the same length. 

* The most convincing method would be to use tests and examinations which 
actually are of the same length; but, since this was not practicable, we made 
the reduction by a statistical procedure of which particulars will be found 
in J. P. Guilford, Psychometric Methods. New York: McGraw-Hill Book 
Company, Inc., 1936. P. 419. 
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TABLE II 


Standard errors of measurement: Tests and examinations 


Test or 
examination 

Standard error of 
measurement • 
(percentage) 

Test or 
examination 

Standard error of 
measurement t 
(percentage) 

E 

4-41 

E 

3-61 

A 

5-15 

A 

377 

Q 

5*30 

I 

519 

I 

5-90 

Qe 

8-i8 

Qe 

5-91 

Q 

8 '9I 

Qa 

865 

Qa 

923 


Comparable for range. 

t Comparable both for range and for length of test. 


The second set of percentages may be taken as giving the fairest 
comparison of the reliability of the various measuring instruments. 
They show that if the standardised tests had an equal chance with 
the examinations, through equHised range and length, they would 
be a much more reliable form of measurement. Their relative 
validity is, of course, another matter. 

In both the tests and the examinations English is more reliable 
than Arithmetic; and the intelligence test is somewhat less 
reliable than the scholastic tests. 

Since there is no variability in the marking of the tests the 
standard error of measurement of y6i per cent, in the English 
test appears to be due to other causes, particularly variation in 
the pupil’s standard of performance. It is tempting, therefore, 
ignoring statistical rigour, to deduct this from the 8*i8 per cent., 
which is the standard error of measurement of Qe; a very rough 
idea of the amount of the unreliability of the English examination 
which is due to variation in the marking is thus secured. This 
would indicate that nearly one-half of the unreliability of an 
ordinary examination is due to variation in the pupil’s performance. 
If this conclusion were even approximately correct it would have 
alarming implications. In the report of the English International 
Institute Examinations Enquiry Committee ^ evidence is afforded of 

* P. J. Hartog and E. C. Rhodes, Tb( Marks of Examiners. London: 
Macmillan & Co., Ltd., 1936, 




RELIABILITY OF THE TESTS AND EX-AMINATIONS 17 

the remarkable variability in the marks given by different examiners 
to the same script. To this, it would appear, there has to be 
added an almost equally great variation arising from the fact 
that the single script is only a chance sample of one out of the 
pupil’s possible performances in similar examinations. 

The practical significance of a reliability coefficient of +*933 
(the raw coefficient for Q) may be expressed in a less technical 
way as follows: 

If a pupil’s true mark in the Qualifying examination were 60 per 
cent., the chance that his actual mark would fall outside the range 

55-65 per cent, is about I in 3, 

50-70 per cent, is about i in 22, 

45-75 per cent, is about i in 370. 

While we have been at some pains to do justice to the newer 
forms of measuring device, the fact remains that in practice the 
Qualifying examination is much longer than the tests. The first 
set of percentages of Table II is therefore the one of most practical 
importance; and it shows that, as things are, a carefully set and 
administered examination can bear comparison, in regard to 
reliability, with standardised tests of intelligence and scholastic 
attainment. It should be mentioned, however, that the tests 
which we selected were shorter than some of those now used 
in Scotland. 



CHAPTER IV 


RELATIONS BETWEEN THE MEASURES USED 
AT THE QUALIFYING STAGE 

The Tests and Examinations 

In approaching the problem of selection for secondary education 
it is of interest to know the extent to which the various tests and 
examinations measure the same factors of ability. Clearly, if the 
correlation between two tests is + l, it does not matter which one 
we use, and there would be no point in using both. The problem 
is affected by the correlations between the measures in another 
way which the following example illustrates. 

Suppose we had four Qualifying tests, A, B, C and D, of which 
the correlations with success in the secondary school (F) were 
rAP=+ 7 > '■bf=+* 6, roP=+-S. ^dp=+* 4 - If ap Education 
Committee desired to use two of these tests for selection purposes, 
which pair should it select ? Taken singly, A and B are the best tests, 
and there would be a temptation to choose them. It is, however, 
quite possible that the best pair to use would be the two worst, 
namely, C and D. This apparent anomaly is due to the fact that 
A and B may be measuring the same aspects of ability, and there¬ 
fore duplicating each other’s work, which would be indicated by 
a high correlation r^. On the other hand C and D may be 
measuring different aspects of the abilities on which success 
depends, and this would be shown by a low correlation roD* The 
choice, therefore, depends partly on the correlations of the tests 
with success and partly on their inter-correlations. 


CORRELATIONS BETWEEN INTELLIGENCE AND THE OTHER TESTS 

AND EXAMINATIONS 

We had two groups, the pupils presented in December 1935 
and those presented in June 1936. We have therefore two 
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estimates in each case, but these were combined, to give single 
correlations^ 

TABLE III • 


Correlations with irttelligence: Tests and examinations 

(Qualifying and age-groups) 


Correlation between I 
and 

Qualifying group 
(N= 2,900) 

• 

Age-group 

(N=l6oo) 

E 

-h-867 

+ ■897 

Qe 

+ •846 


A 

+ 759 

4 --801 

Qa 

4-726 


S 

4--880 


Q 

+ ■855 


Ts ! 

4-791 


T : 

1 

4-715 



• All the differences in correlation within the columns of this table are 
statistically significant. The method used in testing the significance of differ¬ 
ences was to transform the correlation coefficients into z and calculate the 
standard error of the difference of z values. 

In the correlation between intelligence and single tests for the Qualifying 
group the smallest z difference is -074: SE = '026. 

In the correlation between intelligence and combinations of English and 
Anthmetic for the Qualifying group the smallest z difference is *101: SE = *026. 

The z difference for the age-group correlations is *356: SE=:*oi9. 



In combining correlations the procedure used was as follows: 

It was first ascertained whether^the samples on which the separate 
correlations were determined could be regarded as coming from 
equally correlated populations. 


The correlations were then transformed into a, where a = i log, 

The weighted average of the z values was found and re-transformed 


into r. 


Certain special points in regard to combination of correlations are discussed 
in Chapter VII. 

References: R. A. Fisher, Statisiical Methods for Research JForkers. Edin¬ 
burgh: Oliver & Boyd, Ltd., 1936. P. 207. 

P. R. Rider, An Introduction to Modern Statistical Methods. 
New York: John Wiley & Sons, Inc., 1939. P. 105. 
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The correlations are relatively high, a fact which is due partly 

to the wide range of ability in the groups, and partly, perhaps, 

to the great care vidth which the tests and examinations were 
administered. 

The results show: 

That intelligence (by which we mean simply whatever is 
measured by the intelligence tests which we used) is more 
highly correlated -with English than with Arithmetic. 

That intelligence is more highly correlated with the tests than 
with the examinations. 

That intelligence is more highly correlated with the tests and 
examinations than with the teachers* estimates. 

That the scaling of teachers’ estimates ^ in such a way that 
they are comparable from school to school, by a method 
described in Chapter V, increases their correlation with 
intelligence. 

The size of the correlations shows that the intelligence test 
is, to a considerable extent, covering the same mental ground as 
the other tests and examinations; and consequently we cannot 
expect a very large improvement in prediction when it is combined 
with, say, an English test. Also, in view of the fact that the 
correlation is higher with the tests than with the examinations, 
we must not be unprepared to find that a combination of an 
intelligence test with an examination is better for selection purposes 
than its combination with scholastic tests. 

INTELLIGENCE SCORE AND INTELLIGENCE QUOTIENT 

An important practical question is whether we should use the 
intelligence score or the intelligence quotient; and while we can 
settle this most conclusively from the follow-up correlations, 
some preliminary information may be obtained by a study of the 
correlations between the Qualifying data. 

The correlation ri iq is statistically spurious, but it is of some 
interest in this connection, in that it gives us an idea of the extent 
to which the ranking of pupils on intelligence score would be the 

* The correlation with Ts was not, as in the case of the other coefficients 
in Table III, obtained by direct calculation. It was derived from the inter- 
correlations for the senior and junior secondary schools which were calculated 
for determining weightings, a correction for range being applied to make it 
comparable with the other coefficients in the Table. It is based on 2,160 
subjects; and where the accuracy of the method could be tested, the agreement 
with the results of direct calculation was very close. 
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same as the ranking on intelligence quotient. The result is as 
high as +*992.^ 

The fact that this correlation is so high suggests that it would 
matter very little whether we used intelligence score or intelligence 
quotient in selection for secondary education. This does not 
follow: we must have a very high correlation indeed before we 
may draw such a conclusion. 

This important point may be illustrated by actual figures. 
If we have two tests A and B such that rxB= +'992, and if 
r^p= +*700, what can we deduce as to Without making 

assumptions which are normally unsound, it is not possible in such 
a case to determine rjjp exactly; all we can do is to fix limits 
between which its value must lie. When we make the calculation ^ 
we find that tbp must lie between +*604 and +784. Even if 
were as high as +-999, all we could say is that rup must lie 
between +’668 and +731. 

It does not follow, of course, that the correlations will differ 
as much as the above limits permit, and some results for the 
Qualifying group giving actual variations are shown on p. 22^ 
(N = 2,900). 


* This correlation was not calculated directly, but derived by means of the 
following formula from the correlation of intelligence score with age: 



Ml —mean of intelligence scores; <Ti = standard deviation of intelligence scores; 
fl=age. 

This formula assumes that, in expansions, powers of ^ higher than the 

second may be neglected, di representing the deviation of an intelligence score 
.from the mean. 


Reference: C. Udny Yule and M. C. Kendall, An Introduction to tbe Theory 

0/ Statistics. London; Charles Griffin & Co., Ltd., 1937 
Chapter xvi. 




* The Umits for rep are r^r^B =*^ ' ’'ap + '’ab' ap 

Reference: G. Udny Yule and M. G. Kendall, An Introduction to tbe Theory 

of Statistics., p. z8o. 

* The correbtions with IQ were obtained by the method described in 
the lootnote on p, 20. 

The difference between the correlations for IQ and I are barely significant 

etausticaliy. The * differences are-056,-050 and .061: SE=-o28. 
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Correlation with IQ I 

S +-892 +-88o 

Q +*841 +-855 

Ts +767 +791 

Average +.833 +.842 


The conclusion that it makes little difference whether we use 
intelligence score or intelligence quotient will be confirmed later 
by the follow-up correlations. 

CORRELATIONS BETWEEN TESTS AND EXAMINATIONS 

TABLE IV 

Correlations between tests and examinations 
(Qualifying group: N = 2,900) 


1 

Correlation between 

r 

E and Qe 

+ •866 

A and Qa 

+ •810 

S and Q 

+ *898 


The correlations are again high, but even that of nearly *9 
between S and Q leaves ample room for differences in the selective 
value of the two types of measure. One interesting point that 
emerges is that the correlation between E and I is almost exactly 
the same as that between E and Qe. 

CORRELATIONS BETWEEN ENGLISH AND ARITHMETIC 

TABLE V 


Correlations between English and Arithmetic 
(Qualifying and age-groups) 


Correlation 

between 

Qualifying 
group 
(N = 2,900) 

Age-group 
(N = s,6oo) 

Qe and A 

+ 737 


Qe and Qa 

+ 721 


£ and A 

+ 708 

+ 766 

E and Qa 

+ •664 
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The higher correlation for the age-group is due to the wider 
range of ability. 

The correlation between ability in English and ability in 
Arithmetic is a little above 7; but it is interesting to notice 
how much its size depends on the nature of the measures 
used. Even if we take the highest correlation of 7 ^^> clear 
that a knowledge of a pupil’s standing in English gives very little 
guidance as to his standing in Arithmetic. From the graph given 
in fig. 27 ^ it can be seen that the error of prediction would be 
about 64 per cent, of what it would be by pure guess-work. 


CORRELATIONS WITH TEACHERS* ESTIMATES 

TABLE VI • 

Correlations with teachers’ estimates 
(Qualifying group: N = 2,900) 


Correlation between 

f 

Ts and Q 

+ •906 

Ts and S 

+ •866 

T and Q 

+ ■774 


• The first two correlations were obtained by the method described in 
the footnote on p. 20. 

The differences are all significant statistically. The z difference between 
the first two is *188: SE=:*o3o. 


The correlation of nearly *91 between Ts and Q is gratifying, 
but it gives no ground for the belief that the uniform examination 
could be replaced by the teachers’ scaled estimates. Apart from 
the difficulty that the scaling is done on the basis of the examina¬ 
tion marks, the correlation is still far too low. 

Again we see the very significant rise in correlation that results 
from the proper scaling of the teachers* marks; and, as is to be 
expected, the scaled estimates correlate more highly with the 
examination than with the scholastic tests. 


* P. 124. 
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EFFECT OF AGE ON THE CORRELATIONS 

Correlations of the kind with which we are dealing may be 
considerably affected by lack of homogeneity of the groups. The 
Qualifying group is homogeneous in regard to the school class of 
the pupils, but very heterogeneous in regard to age. It is there¬ 
fore necessary to ascertain whether the correlations are inflated 

by the fact that the data correlated have a common correlation 
with age. 

One way in which we can do this is by comparing the correla¬ 
tions for the complete Qualifying group with those for a group 
which is reasonably homogeneous in regard to age; and for this 
purpose we chose the Qualifying pupils who fall within the age- 
group II years 6 months to 12 years 6 months. The correlations 
for the Qualifying group thus analysed into its component parts 
are given in Table VII. 

TABLE VII • 


Analysis of correlations for Qualifying group 


Complete group Middle 
Correlation 10.11-14.9 11.6-12.6 

between 

(N=2,900) (N =2,170) 


Extremes 
under 11.6 and 
over 12.6 

(N= 73 o) 


I and £ 

1 and A 
I and Qe 
I and Qa 
I and Q 
E and A 
E and Qe 
£ and Qa 
A and Qa 
A and Qe 
Qe and Qa 
T and I 
T and Q 

Average 


+ •867 
+ 759 

+ *846 
+ 726 
+ •855 
+ •708 
+ •866 
+ •664 
+ •810 

+ 737 
+ 721 

+ 71S 


+ •878 

+ 773 
+ •843 

+719 

+•856 

+•678 

+ •876 

+•640 

+ •835 

+ 7*4 



• The difference in average correlation between the complete group and the 
extremes is not significant statistically. The other average differences are 
significant. The z difference between the complete group and the middle is 
•031: SE = 'Oo8. Between the middle and the extremes it is -045 : SE = 'Oi2. 
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The Table brings out the interesting result that in a Qualifying 
group the correlation is highest in the middle, and is considerably 
lower in the extremes; that is to say, it is highest where the age- 
scatter is least. The difference is no doubt due to the fact that 
in Qualifying classes the old children are usually dull and the 
young children clever. It is clear, too, that the correlations for a 
Qualifying group are much the same as those for a group which is 
homogeneous in regard both to age and to school class. 

The above results can be shown from a different angle by using 
the correlations tvith age. We should expect a negative correla¬ 
tion in the extremes, but, in fact, all the correlations with age for 
the Qualifying group are negative. A small selection, calculated 
by Dr McIntosh, is given below: 


Correlation ^ between 

Middle 

Extremes 

age and 

(N=2,2I7) 

(N= 737 ) 

Qe 

- -086 

-•129 

Qa 

- 112 

- -129 

Q 

- -102 

- -156 

Average 

- -100 

-•138 


We may conclude from this evidence that even in that part of 
a Qualifying group which is fairly homogeneous in regard to age 
the correlation with age is negative. This means that the older 
pupils are, on the whole, slightly poorer in ability and attainment 
than the younger. 

USE OF PARTIAL CORRELATION METHODS 

By the method of partial correlation we can find what the 
correlations would be if all the pupils were of the same age; and, 

I All the correlations here are significantly different from zero. The prob¬ 
ability that they could be obtained by random sampling from a population of 
zero correlation is of the order P = •0003. 

_ In the few cases where it was necessary to test the significance of our correla¬ 
tions the formula used was 

rVn-z 

VT-r* 

The differences between the columns arc not significant statistically. The 
a difference for the average is .039: $£=.024. 

References: R. A. Fisher, Statistical Methods for Research Workers^ pp. 195 

and 205. 

P. R. Rider, An Introduction to Modern Statistical Methods, 
pp. 83 and 93. 
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as would be anticipated from the above results, the differences 
from the raw coefficients are extremely small. Out of five cases 
calculated by Dr McIntosh the largest difference was *002. 


EFFECT OF DIFFERENCES IN SCHOOL CLASS ON THE 
CORRELATIONS FOR THE AGE-CROUP 

The Qualifying group is homogeneous in regard to school class 
but heterogeneous in age; the age-group, on the other hand, is 
relatively homogeneous in regard to age, but heterogeneous in 
regard to school class.^ Our data provide us with an opportunity 
for making an analysis of the correlations for such a group; and, 
while this is not of much importance for our selection problem, 
it is of considerable interest in connection with the interpretation 
of correlation coefficients. In this case we shall count the pupils 
in the Qualifying class as the middle, and those in the classes above 
and below as the extremes. 

TABLE VIII 


Analysis of correlations for Qualifying and age-groups combined 


Group 

N 

Average of 
three 

correlations * 

Age-group outside Qualifying classes . 
Complete age-group ..... 
Combined age-group and Qualifying group . 
Qualifying group in age-group . 

Complete Qualifying group 

Qualifying group outside age-group 

3.414 

5,608 

6,335 

2,194 

2,921 

727 

+ •844 
+ •821 
+ •817 
+ •802 
+ 778 
+ 776 


• TtB, riA and 


While in the Qualifying group the correlation in the extremes 
is lower than that in the middle, or relatively homogeneous part, 
the opposite holds for the age-group.^ The position is, therefore, 

1 Cf. pp. 2 and 3. 

■ The difference is statistically significant. The « difference is *131: 
SE=*oi6. 
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that at the Qualifying stage in Scotland the inclusion of the old 
and young pupils in a group which is homogeneous in regard to 
school class lowers the correlations; but that the inclusion of 
pupils from higher and lower classes in a group which is homo¬ 
geneous in regard to age raises the correlations. In this connection 
it should be mentioned that in the case of the age-group the 
standard deviation in the extremes is higher than the standard 
deviation in the middle, the opposite being true of the Qualifying 
group. 

From the Table we see that the correlation for an age-grouping 
is higher than that for a class-grouping, the difference in correla¬ 
tion being about *04.^ 


The Data from the Rating Cards 

INDUSTRY 

(In collaboration with Miss J. I. W. Robertson, M.A.) 

The Qualifying teachers were asked to grade all pupils in regard 
to industry, using a five-point scale in which the significance of 
the letters was as follows: 

A Exceptionally high. 

B High—decidedly above the average. 

C Average. 

D Low—decidedly below the average. 

E Exceptionally low. 

The leaflet of instructions gave the percentages of pupils who 
would fall into the various categories in a normal class. 

Using the June group. Miss Robertson calculated correlations 
between these industry gradings and intelligence quotient, Qualify¬ 
ing examination marks, and health. She dealt with boys and girls 
separately. 

All correlations which we have calculated for data on five-point 
scales have been corrected for broad grouping.* 

* The difference is statisticaUy significant. The » difference is -iiq: 
SE=.oi3. ^ 

References: K. Pearson, “On the Measurement of the Influence of Broad 

Categories upon Correlation,” Biotneirika^ vol. ix, 1913, 
p* 116. 

K. J, Hohxngtr^ Statistical Methods for Students in Education, 
New York: Ginn Sc Co., 1928. Chapter xiv. 
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TABLE IX* 


Industry: Correlations with IQ, Q, H 
(June Qualifying group) 


Correlation 

between 

Boys 

Girls 

All 

industry 







and 

N . 

r 

N 

1 

f 

N 

f 

IQ 

953 

+•598 

987 

+ •647 


+•622 

Q 

931 

+•568 

956 

+ 664 

1,887 ! 

+•616 

H 

736 

+•112 

763 

+ •140 


+■126 


* The correlation with health is signlBcant statistically, P being far below 
•01. As far as the differences between boys and girls are concerned, the 
difference between the correlations with Q is significant; z difference is 
• 155: SEs=- 045. In the other two cases the differences are not significant, 
though that for IQ is nearly so; * difference is -080: SE ='045. 


It may be that there is a tendency for teachers to ascribe a high 
degree of industry to a child who is doing well in class, while his 
success may be due rather to superior ability or home encourage¬ 
ment. On the whole, however, we felt that the industry gradings 
were much more reliable than those for interest, which we shall 
discuss presently. 

Taking them at their face value, they show a correlation of about 
•6 both with intelligence and scholastic attainment and only a 
small positive correlation with health. From our point of view 
the results offer hope that industry may be of some value in fore¬ 
casting success, since its correlation with the other measures is 
such as to suggest that it may cover a part of the field which they 
leave untouched. Perhaps the result that gives most food for 
thought, however, is the size of the correlation with IQ. 

It is interesting, too, to notice that the correlations for the girls 
are all higher than those for the boys, though the difference is 
significant statistically only with Q. Some further data on the 
question of the difference in industry between boys and girls are 
given in Table X. 

The Table gives the percentages falling into the various categories. 
They are comparable as between industry and Qualifying examina- 
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tion marks, for the latter distribution was cut in such a way that 
the dividing lines between the categories are at the same o distance 
from the mean in both cases. The means at the foot of the 
columns are found by giving five points for an A grading, four 
points for a B and so on, and may be taken as giving a rough 
comparative measure of the quality of the groups. 

TABLE X 


Industry: Comparison of boys and girls 
(June Qualifying group) 


Category 

Boys 

(N= 93 i) 

Girls 

(N= 956 ) 

Ind. 

Q 

Ind. 

Q 

A 

30 

20 

3-0 

2-9 

B 

26-5 

300 

268 

305 

C 

53-6 

486 

568 

50-4 

D 

150 

169 

II -5 

14-8 

E 

1-8 

2-6 

1-9 

1-2 

Mean 

3-14 

312 

00 

3-19 


The differences between boys and girls are statistically significant 
both in industry and in Qualifying examination marks, the former 
difference being nearly four times its standard error. On the 
whole, therefore, the results support the view that girls are more 
industrious than boys, though the difference is not very great. 
A closer study of the table shows where the difference mainly lies. 
The good boys are nearly as industrious as the good girls, but the 
boys of poorer ability are less industrious than girls of the same 
type. 

PUPILS* INTERESTS 

(In collaboration with Mr G. A. Crabb, B.Sc.) 

Predictive Value of Teachers^ Interest Gradings 

With a view to exploring the value, in guidance and selection, 
of information as to the direction of the pupils’ interests, we asked 
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the primary teachers to grade each Qualifying pupil on a five-point 
scale in regard to interest and attainment in the three main groups 
of school subjects—humanistic, realistic and practical. The 
meanings of the letters were the same as in industry. 

Mr Crabb made a study of these interest gradings; and, while 
his rwults did not encourage us to proceed with the statistical 
working-up of this part of our data, he continued his work on the 
subject of children’s interests until his entfy on war service. The 
following is an account of the part of his investigations which bears 
upon the present Inquiry: his further results he will, no doubt, 
publish independently. 

The group chosen was that presented in June, which showed a 
good distribution of pupils among the five categories. When, 
however, the gradings were compared with those for attainment 
there was ground for suspicion that the agreement was greater 
than was probable on the basis of a true assessment of interest. 
Mr Crabb made the comparison in two ways: by finding the 
numbers of pupils who came into the same category both for 
interest and for attainment, and by finding the correlation, 
corrected for broad grouping, between the interest and attainment 
gradings. By the first method he found that no less than 84 per 
cent, had the same grading in both. The correlation between 
the interest and attainment gradings in humanistic subjects was 
+ -94. 

Mr Crabb planned an interesting subsidiary experiment to find 
how far the teachers’ estimates of interest corresponded with 
the children’s own ideas on the subject. A new group was formed, 
consisting of the Qualifying pupils in five representative schools. 
These pupils were graded by the teachers in exactly the same way 
as the Inquiry group had been. They were then given a simple 
Interests questionnaire. 

Sixteen school subjects or parts of subjects (for example. Poetry, 
Composition, Spelling) were listed, and the pupil was asked to 
indicate by underlining whether he liked the subject, disliked it, 
or was uncertain whether he liked or disliked it. For purposes of 
scoring, a like was credited with three marks, a not sure by two 
marks, and a dislike by one mark. The subject selected as favourite 
was given an extra mark. 

The school subjects were then divided into humanistic, realistic 
and practical, so that the pupil’s score in each group could be 
obtained. By cutting the distribution of scores in such a way that 
the border-lines were at the same a distance from the mean both 
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in the teachers’ gradings and in the pupils’ own scores, a comparable 
five-point grading, made by the pupUs themselves, was secured. 


TABLE XI 

Interest; Comparison of gradings made by teachers with those made 
by pupils (special experimental group: N-227) 


Group of subjects 

Correlation between 
teachers’ and pupils* 
gradings 

Percentages of pupils 
in same category in 
both gradings 

Humanistic 

+ •26 

do 

Realistic . 

+ •46 

440 

Practical . 

+ ■13 

39-6 


The average correlation between the teachers’ gradings in 
interest and attainment was +’95, which is very close to that 
obtained for the Inquiry group. The average correlation between 
the pupils’ gradings of their own interests and the teachers’ grad¬ 
ings for attainment was + ’40. 

These results show that there is great disparity between the 
teachers’ estimates of children’s interests and the children’s own 
ideas; and they tend to confirm the suspicion that the teachers’ 
gradings were largely influenced by the pupils’ attainments. That 
such may be the case is no reflection upon the care with which the 
teachers made their estimates: the fault lay in our technique, 
which set them an impossible task. We decided, however, that 
instead of calculating follow-up correlations for the interest data 
on the rating cards, Mr Crabb should follow up the pupils in his 
experimental group with a view to comparing the prognostic 
value of the teachers’ and pupils’ gradings. This project had to 
be abandoned on account of his entry upon war service. 

Pupil/ Order of Preference of School Subjects 

The method of scoring used by Mr Crabb enabled him to deter¬ 
mine comparative interest scores for the various school subjects, 
and, though the number of pupils was rather small, fig. 4 is 
presented as a contribution to the mounting evidence on this 
topic. It gives the order of preference for a group of pupils at 
the Qualifying stage in a Scottish industrial City, and shows also 
the range of variation in the placing of the subjects by individual 
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pupils. The centre points give the mean placing of the subjects, 
and the horizontal lines cover a range of ±3cr. 


PAINTING 


^_ 

PHYS.EOUCN. 


Q 

DRAWING 


Q_ 

HANDWORK 


p 

ARITHMETIC 


— 0 — 

READING 


— 0 — 

HISTORY 


■ 0 —- 

POETRY 


_ 0 — 

WRITING 


- 0 - 

GEOGRAPHY 


0 

SPELLING 


-0 

NATURE STUDY 


0 

COMPOSITION 


_Q 

MENTAL ARITH. 


0 - 

interpretation 

• 

_o_ 

GRAMMAR 



1 

11 1 — 

130 170 

210 250 290 330 


Fig. 4.—Pupils’ preferences in school subjects. 


The vertical line between 170 and 210 divides subjects where the 
number of likes is greater than the number of dislikes from those 
where the reverse is the case. It could hardly be assumed that this 
gives a border-line between absolute liking and dislike; one suspects 
that the child gives his judgment with the reservation ‘as far as 
school subjects go.’ Still, the results confirm the view that 
Grammar is a distasteful subject to the majority of pupils at 
this age. Yet even here, as a cautious boy put it, ‘everything 
depends on the teacher’; in one class the gradings for Grammar 
were: like^ 60 per cent., not sure^ 40 per cent., dislike^ o per cent., 
favourite subject^ 30 per cent. 

Comparison of Interests of Boys and Girls ^ 

The number of pupils in this experiment was too small to yield 
reliable results, but, generally speaking, they give no ground for 
the belief that there is any significant difference between the 
educational interests of boys and girls at this stage. There were 
small differences in the order of preference of school subjects but 
only one or two stood the test of statistical significance. Nor, as 
is shown in Table XII, is there any statisticaUy significant difference 
between boys and girls in the three main groups of subjects. 
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TABLE XII 

Interest: Comparison of mean scores of boys and girls in humanistic, 

realistic and practical subjects 
(special experimental group: N =227) 


1 

Group of subjects 

Boys 

(N = 116) 

Girls 
(N = iii) 

Humanistic . 

2*31 

2-39 

Realistic 

2 46 

2*42 

Practical 

2-75 

2-83 


As might be expected, both groups show the greatest preference 
for the practical subjects, and a slightly greater preference for 
the realistic than for the humanistic subjects. 


2 




CHAPTER V 

TEACHERS’ ESTIMATES 

(In collaboration with Mr John T. Bain, M.A., B.Sc.) 

Variation in Standard from School to School - 

On a preliminary survey the teachers’ estimates appeared to have 
the following characteristics: 

They were generally too high. 

They tended to over-estimate the attainments of the weak 
pupils and under-estimate those of the good pupils. 

The standard varied considerably from school to school. 

The first of these features is not very serious from the point of 
view of follow-up correlations. The second decreases the weight 
given to teachers’ estimates when they are combined with other 
measures having a larger standard deviation. The third is the 
most important, for it would lower the correlations considerably.' 

To ascertain the extent of the variation in standard, Mr Bain 
analysed the data for a representative group of twelve schools 
comprising about 850 pupils in all. 

At the moment we are concerned only with the columns in 
Tables XIII and XIV which are headed Te, Qe, Ta and Qa. 
They show that in English the teachers’ estimates are, on the 
average, 15 per cent, higher than the scores made by the same 
pupils in the uniform examination; in Arithmetic they are 25 per 
cent, higher. 

In English the teachers’ estimates are much less widely scattered 
than the examination marks, the average standard deviations being 
12*3 and 18-4 respectively; in Arithmetic the difference in scatter 

is very much less. , , 

The variations in standard from school to school are very 

striking. In English the teachers’ estimates for School 4 are on 
the average 1-9 per cent, above the mean mark scored in t e 
examination; in School 9 they are 30*2 per cent, higher. The 
difference in standard of marking between these two schools is 

> Cf, Chapter VII. 
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therefore more than 28 per cent. In Arithmetic the maximum 
difference in standard is 22 per cent. 


TABLE XIII • 


Teachers’ estimates in English: Comparison with marks in English 
scored by the same pupils in the Qualifying examination 



It is thus couHrmed that teachers’ estimates have the three 

fr mird'^SJ* » has to be kpt 

m mind that if the estimates are higher than the examination 

marks it does not foUow that it is the teachers’ standard that is 

wrong, w^e the difference in scatter may be partly due to the 

cLTZf --g" of Ve number of 


‘ Since a? 


(x+T+£+.. .)-2a|+z5> 


<y 

XT X T 
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TABLE XIV 


Teachers’ estimates in Arithmetic: 
scored by the same pupils 


School 

Ta 

M 

a 

I 

690 

243 

2 

69-5 

16-5 

3 

71-3 

13-4 

4 

72-8 

16-5 

5 

72-0 

19-4 

6 

68-5 

III 

7 

791 

13-8 

8 

74-5 

20*3 

9 

77-4 

229 

lO 

62-6 

22-8 

II 

58-6 

107 

12 

65-8 

181 

Average . 

70-4 

17*6 

Slumped schools 

70-4 : 

i8-8 


Comparison with marks in Arithmetic 
in the Qualifying examination 


Qa 

Tas 

^QftTa 

M 

<7 

M ' 

a 

52-4 

20*9 

52-8 

22-0 

+ •88 

35*9 

' 160 

37-0 

17-2 

+75 

51-6 

17-3 

512 

i7'6 

+•81 

58-5 

i6*3 

57-6 

15-8 

+78 

50-4 

i8-i 

476 

i6-6 

+•80 

SS'S 

157 

537 

140 

+70 

517 

187 

49-6 

150 

+75 

40-3 

172 

42-6 

20-8 

+•82 

+76 

425 

147 

425 

16-4 

37-0 

22-0 

388 

24-8 

+ '82 

26-6 

13*3 

23-0 

12*3 

+74 

381 

19-9 

40-1 

21-6 

1 

1 

+ 01 

45-6 

17-9 

45-4 

1 

i8-3 

+77 

45-6 

19-9 

45-4 

20*1 

+73 


Comparison of Variations in Standard for English 

AND Arithmetic 

The question is whether a school that marks 
rloolttee marb rUdev'scTtrered Tn English shows a 

-.x ™ --S 

““c., a.. “ » f£ 

p;; - S'; S 
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average, and made a corresponding correction in the standard 
deviations.' 


TABLE XV 


Teachers’ estimates in English and ,\rithmetic: Comparison of 

variations in standard of marking 


School 

M 

0 

Te-Qe 

Ta -Qa 

Te-Qe 

Ta-Qa 

I 

+ 8i 

+ 6-8 

+ -6 

+ 3 -+ 

2 

+ i8-3 

+ 238 

- -I 

+ -s 

3 

+ 2-9 

+ 9-9 

- -7 

- 3-9 

4 

+ 1-9 

+ 4*5 

- -5 

+ -2 

5 ' 

+ 6-8 

+ II-8 

+ -2 

+ 1-3 

6 

+ 131 

+ 3-2 

-1-6 

- 4-6 

7 

+ i6-6 

+ 176 

- 2-3 

- 4-9 

8 

+ 19-4 

+ 24-4 

- -8 

+ 3-1 

9 

+ 30-2 

+ 25’I 

+ 2-5 

+ 8-2 

10 

+ 22-8 

+ 15-8 

+ 4-6 

+ *8 

11 

+ 21'0 

+ 22-2 

- 1-7 

- 2-6 

12 

+ 229 

+ 17-9 

- 2-7 

-1-8 


The answer to our question is, therefore, that we should make 
serious errors if we assumed that a school which marks leniently 
in English marks equally leniently in Arithmetic, or that a school 
whose marks are widely scattered in English shows a correspond¬ 
ingly wide scatter in Arithmetic. 


Correspondence between Orders of Merit in Teachers* 
Estimates and Uniform Examination 

Mr Bain calculated correlations between teachers’ estimates and 
Qualifying examination marks for each school, for English and 
Arithmetic separately. These are given in the last columns of 
Tables XIII and XIV, from which it will be seen that the average 

^ It should be noted that the standard deviations for the slumped schools in 
Tables XIII and XIV have a different meaning from the average standard 
deviations. For this correction we use the average standard deviations. 
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correlations for English and Arithmetic are +-82 and +'77 
respectively. The difference is statistically significant, so we may 
conclude that there is a closer correspondence in English than in 
Arithmetic.^ In neither subject, however, is the correspondence 
as close as we might expect in marks which purport to measure the 
same abilities. 


Methods of Reducing Teachers’ Estimates from different 

Schools to the same Standard 

The above results show that teachers’ estimates, as they stand, 
should not be used in selection and guidance. Before using them 
some means must be found for making them comparable in standard, 
and there are two ways in which this improvement might be 
effected. 


guidance to teachers as to standard for estimates 

The first line of attack is to devise ways in which teachers could 
be helped to use a uniform objective standard of marking, but all 
that we could attempt in this direction was a preliminary investiga¬ 
tion of one of the possible methods.* We asked the teachers to 
give estimates on a five-point scale where each category had a 
definite significance, the percentages of a normal class that would 
fall into each category being indicated in the leaflet of instructions. 
The gradings were made for attainment only, taking no account of 
the pupils’ ages.* 

Mr Bain investigated the question whether these letter estimates 
were on a more uniform standard than the percentage marks. He 
took the five-point gradings in attainment in humanistic and 
realistic subjects, determined the points on a percentage scale 
which corresponded to the means of various categories, and thus 
found a rough measure of the means and standard deviations for 
the various schools. These are shown in Table XVI. 

We cannot attach much importance to the fact that the averages 
of the means on the five-point scale are much closer to the corre- 


• a difference is *136: SE=*o49. 

• A useful discussion on this subject will be found in the report of the 
Swedish Committee of the International Institute Examinations Inquiry— 
r. Wigforss, 7 be Entrance Examination in view of Later School Performances. 
Stockholm; P. A. Norstcdt & Sflners Fdrlag, 1937. P no. 

• For the significance of the letters see p. 27* 
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TABLE XVI 


Teachers’ estimates on five-point scale: Improvement in 

uniformity of standard 



English 

Arithmetic 

School 

Te 

(marks) 

Te 

(5-point 
scale) 

Qe 

Ta 

(marks) 

Ta 

(5-point 

scale) 

Qa 

I 

6+7 

55-2 

566 

690 

58-6 

52-4 

2 

604 

45-8 

421 

695 

49-8 

35-9 

3 

6+-9 

442 

620 

71-3 

5 J -4 

SI-6 

4 

63-9 

54-8 

620 

728 

60-0 

58-5 

5 

67-5 

53-1 

60 7 

720 

520 

504 

6 

65-3 

517 

52-2 

68-5 

527 

55-5 

7 

74-7 

5 > 4 

58-1 

791 

5»-4 

5*7 

8 

652 

50-9 

45-8 

74-5 

53-0 

40-3 

9 

76-1 

50-4 

45-9 

77-4 

509 

42-5 

10 

611 

466 

383 

626 

44-2 

370 

11 

605 

46-5 

39-5 

S8'6 

45-0 

26-6 

12 

672 

48-2 

++■3 

65-8 

509 

38-I 

Average M 

66-1 

49-6 

511 

704 

5*7 

45-6 

Average a 

12*3 

I +-3 

203 

176 

* 5-4 

19.9 


spending figures for the uniform examination, for this depends on 
the statistical adjustment. To find the improvement in standard 
of marking, both forms of estimate were reduced to the same means 
as the corresponding examination marks and the average differences 
calculated. These are given in Table XVII,‘ from which it will 
be seen that the improvement by the use of the five-point scale 
is !•! per cent. That this is not sufficient is confirmed by a 
scrutiny of the results for the individual schools. In English, with 
the five-point scale. School 3 under-estimates by 17-8 per cent., 
while School 10 over-estimates by 8*3 per cent. 

Mr James C. Kidd, M.A., B.A., investigated the correlations 
of the five-point estimates with later success in the senior secondary 

‘ P- 43 * 
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courses, and found that they^ were all lower than the corresponding 
correlations for the percentage estimates. 

We conclude that a considerable improvement in technique 
would be necessary before the five-point method would yield 
results that could profitably be used in practice. 

THE SCALING OF TEACHERS* MARKS 

The second method is to take the teachers’ estimates as they 
are and to reduce them to comparability by statistical procedure. 
For this purpose it is not sufficient to use standard scores, since 
this would assume that the pupils of all the schools had the same 
mean level of attainment. Clearly we must transform the teachers’ 
marks in such a way that the mean and standard deviation of the 
transformed marks for any school will be the same as the mean and 
standard deviation of the marks made by the pupils of that school 
in a uniform test or examination in the same subject. This can 
be done by the transformation ^ 

Ts = Mq + ^(T - Mt) 

<7x 

and this, in our view, is the soundest method to adopt. It is, 
moreover, simple enough in practice. Since the equation is linear, 
the constants are easily calculated and a transformation Table can 
be quickly made. 

It has, unfortunately, one disadvantage: in certain cases the 
transformed marks fall outside the limits of the percentage scale. 
The statistician would readily reconcile himself to this, but it is 
doubtful whether an Education Committee would viewwith favour 
any method which ascribes a negative mark to a pupil. Such 
negative marks are, of course, not so absurd as they seem at first 
sight. They do not mean that the pupil has less than a zero 
knowledge of the subject, merely that he is a certain distance 
below some arbitrarily chosen point of reference. Apart from 
the possible misunderstandings, however, there is a practical 
difficulty in the use of negative scores in that they are a fruitful 
source of error. 

Method used in the Inquiry 

We experimented with various methods of overcoming the 
above difficulty, keeping in mind the fact that the system used 

» JC. J. Holzinger, Statittical Metbodi for Studenti in EdticatioHj p. lai. 
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should be one that could be carried out quickly by the ordinary 
clerical staff of an Education Office; and in the end the following 
procedure was adopted. VVe took the teachers* estimates and the 
examination marks for a school and equated the quartiles and ends 
of the ranges, giving a graph of the type shown in fig. 5, A 
separate graph must of course be made for each school. If the 
raw mark falls at P we read the transformed mark at Q. 



Fio. 5.—Graph for scaling teachers’ marks. 

While the equation of the quartiles is not open to serious 
statistical objection, it was with some hesitation that we decided 
to use the highest and lowest scores. We must have two fixed 
points outside the quartile range Q1-Q3 if we wish to avoid 
negative s-ores, yet in using the ends of the ranges we felt that we 
were too much dependent upon accident: a pupil who had made 
an exceptionally low score in the examination would pull down 
the marks of all pupils below the lower quartile. The procedure 
was, however, given a very searching test through its application 
to the whole Qualifying group, and our justification for offering 
It as a practical solution of the difficulty is that with no pupil 
aid It appear to operate unfairly. ^ 
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Difficulties in the Use of Scaling Systems 

The main practical difficulty is that tio scaling system can work 
satisfactorily with small numbers, and these may occur in two 
cases: 

In country districts the number of Qualifying pupils in any 
one school may be very small. 

In urban systems where the three-track plan is used the 
numbers in the lowest section may be small, even in a 
school of considerable size. 

Of these we had to contend only with the second, and the 
difficulty did not prove to be serious. We ascertained what pro¬ 
cedure had been used in making the estimates; and fortunately 
most schools had given the marks for the different sections on the 
same standard. In a small number of schools the C section pupils 
were marked more leniently, and the estimates had to be scaled 
separately. 

The difficulty arising from the three-track plan can be overcome 
by requiring the teachers to give their estimates on the same 
standard for all pupils. For the small rural school the system m 
scaling is quite impracticable; and, unless uniformity of standard 
can be secured in other ways, the use of teachers* estimates should 

be abandoned. ., 

A third possibility of breakdown in the scaling system would 
not arise if our final recommendations are accepted. In certain 
Areas a special examination is held for admission to secondai^ 
schools or for the award of bursaries, and it may happen that the 
number of candidates from a particular school is very small. We 
believe, however, that any sound system of guidance and selection 

at the Qualifying stage demands the examination of the complete 

Qualifying group, by which we mean all the pupils in the Qualifying 
classes. 


Improvement effected by Scaling 

The improvement effected by scaling can be inferred from 
Tables XIII and XIV, where the school means of 

transformed estimates are given. The standards 
are now closely in conformity with the marks scored by ^e r 
pupils in the uniform examination, and the scatter is approxi 
matelv that given by the examination. If we make all ^e averag 
means tke fame, we obtain a fair comparison of the average 
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difference between the school means and the examination means 
for the three types of estimates with which we have experimented. 


TABLE XVII 


Teachers’ estimates: Average difference between school means and means 
of Qualifying examination marks for various types of estimate 


1 

Type of estimate 

Average difference (per cent.) 

English 

Arithmetic 

Unsealed percentages . 

7-4 

6-7 

Five-point scale .... 

6-3 

5-6 

Scaled percentages 

»-5 

1-6 


Perhaps the most important result of the scaling is that the large 
differences in standard have entirely disappeared. 


Separate Scaling for English and Arithmetic 

So far we have examined the procedure where English was 

scaled on English and Arithmetic on Arithmetic. This involves 

a good deal of labour, which could be halved if we simply scaled 

the totals for the two subjects. To consider whether this would 

be legitimate or desirable we have to introduce a somewhat 
cumbrous notation: 


Let Te = teachers’ raw estimates in English, 

Tes = teachers’ estimates in English scaled on English, 
and similarly for Ta and Tas, 

T = teachers’ raw estimates in EngUsh + Arithmetic. 
1 IS therefore equal to Te+Ta. 


We can arrive at the teachers’ scaled estimates for English + 
Arit^etic m two ways. Te may be scaled on Qe, giving Tes 
^d Ta on Qa, giving Tas. This method gives Ts = Tes+Tas’ 
We may, on the other hand, scale Te + Ta on Qe + Qa, obtaining 
a result which we shall denote by Ts. ^ 

Our problem is to ascertain whether, for purposes of guidance 
nd selection at Ae Qualifymg stage, it would be legitfmate or 
desirable to use Ts instead of Ts. This is a question Twhi^ 
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the most satisfactory answer would have been obtained by using 
both methods of scaling and by calculating follow-up correlations 
for the resulting scaled estimates. This was not done, partly 
because of the labour involved, and partly because we have other 
reasons, which we shall state presently, for preferring Ts to Ts. 
Without actually scaling we can roughly deduce the predictive 
value of Ts as compared with Ts. The method used has a some¬ 
what mathematical appearance; and as it is too long for a foot¬ 
note it has been put into an indented section which the general 
reader is advised to omit. 

Since the method of scaling adopted is an approximation 
to the exact way of making the mean and standard deviation 
of the teachers’ estimates equal to the mean and standard 
deviation of the marks in the uniform examination, we may 
take it that 

Tes = MQ.+^“(Te-Mx.) 

OXe 

It can then be shown that, for a single school, 

. <^Qe«^Te + + (<TQe<rTB + OQ»ax^^TeTa_ 

TaT» " ^{axo + + 2rxeXa<yTe<^TB} V {<^Qe + <^Qb + ZrieXftO'QeO^Qa/ 

which would not normally be equal to unity. 

For one of the schools in Mr Bain’s sample “ *4 ' 9 ^* 
This is a fairly high correlation, but, as we have seen, we 
cannot without other evidence say that it would not matter 
which we used for prognostic purposes. 

It can also be shown that 


rxip — 


and 




_ gXe^FXe + O'Ta^FTft _ 

V *4 + 2rjgXaO'Te*^Ta} 

Crpe^FTe ~4 O’Qa^FTa 


\/{oq^ + CTQa + 2r^eXa°^Qo*^QaJ 

From our other data we know the limits between which 
and Tm are likely to vary. We can, therefore, 
SeXid compisons usfng the^schools in Mr Bain’. 
”mple Ind experimenting with different values of the 
co“relatiLs. Such experiments are not a very rigorous 
way of settling the question, but they lead to the foUowmg 

conclusions: 
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In the great majority of cases rjaF would be higher 
than but m exceptional circumstances r^p could be the 

greater. . i j i 

For a single school of reasonable size would rarely 

exceed by more than -03: the difference would normally 

be about '02. 

Our conclusion is that the correlation with later success of 
teachers’ estimates scaled separately for English and Arithmetic 
would be higher, probably by about -02, than that for estimates 
scaled for English and Arithmetic together. A stronger case for 
the first method can be made out on other grounds, of which the 
most compelling is the need for satisfactory separate estimates of 
the pupil’s ability in the two subjects. 

Scaling Teachers’ Estimates on Intelligence Test Results 

The educational importance of this problem is much greater 
than at first sight appears. It lies in the fact that, if we could 
legitimately scale teachers’ estimates on the intelligence test, and 
if a combination of an intelligence test and teachers’ scaled 
estimates gave a high correlation with later success, we could 
avoid all the educational evils of uniform external examinations 
at the Qualifying stage. 

At this point we can deal only with the question whether the 
method would be legitimate, and we take first the simpler problem 
of the scaling of the total marks in English + Arithmetic. For this 
purpose we use as our standard of comparison the teachers’ marks 
scaled by the soundest possible method, that is, on Qualifying 
examination scores. With them we must compare the teachers’ 
estimates as scaled, say, on IQ, which we shall call Tsi. 

Mr Bain investigated this point with his sample of twelve 
schools. 

Glancing along the rows of Table XVIII we can see that scaling 
on IQ makes a considerable improvement, but not so great as 
that made by scaling on Q. Using the method previously adopted 
we compare the improvements in Table XIX. This is a Table 
which we present with considerable regret. It shows that the 
method of scaling upon IQ cannot be regarded as sound. Even 
in the small sample of twelve schools injustice to the extent of 
over 12 per cent, could be done to considerable numbers of pupils, 
and it seems unfair to use a system which has a known error of 
this size. 
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TABLE XVIII 


Teachers’ estimates scaled on Q and on IQ 


School 

Q 

Ts 

Tsi 

T 

I 

55*6 

55 '* 

54-9 

677 

2 

39-8 

40-0 

44-0 

64-0 

3 

57*5 

569 

620 

67-2 

4 

6 o*4 

587 

56-0 

68-6 

5 

56-4 

55-8 

54-8 

690 

6 

53-8 

529 

S 2-0 

66*5 

7 

55-6 

53-6 

541 

75*4 

8 

43*6 

447 

51-6 

697 

9 

441 

43*8 

49*9 

77-0 

10 

37-8 

37-9 

43-2 

6i-o 

11 

34-6 

340 

415 

59-0 

12 

417 

44-4 

46-8 

66-2 

Average M 

48-9 

487 

51-3 

677 

Average a 

i6-8 

i6-6 

I5-I 

14*0 


TABLE XIX 


Teachers’ estimates scaled on Q and on IQ: Average diflference between 
school means and means of Qualifying examination marks 


Type of estimate 

1 

Average difference 
per cent. 

Maximum error • 
per cent. 

T 

6-9 

247 

Til 

3-8 

12-4 

Ts 

•9 

47 


• Maximum error=sum of largest positive and negative differences. 


Mr Bain calculated the following correlations with teachers 
estimates scaled on the Qualifying examination and on the 
intelligence test for his group of 842 pupils: 
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Correlation of Q and T (average) +-836 

Q and T (slumped) + 787 

Q and Ts (slumped)+-883 
Q and Tsi (slumped) +*857 

Ts and Tsi (slumped) +-971 

The most interesting of these correlations is the last. Despite 
the fact that the teachers’ estimates scaled in these two ways 
correlate to the extent of ■97i» we should, as we have seen, do an 
injustice amounting to as much as 12 per cent, to certain pupils 
if we used the method of scaling on IQ. The middle pair show 
that the marks scaled on the examination correlate more highly 
with the examination than the same marks scaled on IQ. The 
first pair give another example of the way in which correlations 
are lowered when we slump together schools where there is a 

variation in standard of marking. 

Another disadvantage of the intelligence test as a scaling basis 
is that we cannot obtain reliable estimates in English and Arith¬ 
metic separately without assuming that the relative positions of 
the school means are the same in both subjects. The unsoundness 
of this assumption is clearly brought out by Table XV and by the 
following correlations between school means for the June Qualify¬ 
ing group, comprising 36 schools; 

Correlation between school 
means 

IQ and Qe + ’88 
IQ and Qa + ’86 
Qe and Qa +*89 

These are far from high enough to justify a scaling of English or 
Arithmetic on IQ. 

Conclusions 

We conclude, therefore, that the only satisfactory procedure is to 
scale the estimatesseparatelyforEnglish and Arithmetic, and that the 
estimates should be scaled upon the results of a uniform examination. 

Steps should at the same time be taken to ensure that all the 
teachers make their estimates on the same basis. The instructions 
should make it clear whether the estimates are to be measures of 
the pupils’ attainments at the time of examination, or whether 
they are to take account of evidence of progressive improvement or 
deterioration; abo whether allowance is to be made for ability, 
age and personal qualities. 



CHAPTER VI 


COMPARABILITY OF SUCCESS MARKS IN THE 

SECONDARY COURSES 

The Measxhie of Success 

To compare the prognostic value of the various measures used at 
the Qualifying stage we require a criterion by which they can be 
judged, and this should, strictly speaking, be the child’s ability 
to profit by the secondary course. For scientific investigation, 
however, we required one that could be expressed in exact 
quantitative terms; and we used the child’s success. The two, 
we grant, are not quite the same: a child may profit much more 
from a secondary course than his success mark would indicate. 
Yet, from the point of view of the child, the parent and the 
Education Committee, it is success that is the more important. 
Moreover, the pupil who consistently scores low marks in his 
secondary school examinations has much on the debit side of his 
‘ability to profit’ account, no matter how much he has on the 
credit side in the way of response to cultural stimulus and social 
contacts. 

Accepting success as our criterion, we have still to determine 
what we are to take as its measure; and after careful consideration 
we decided to use the weighted average of the marks in all the 
subjects of the secondary course. 

Doubt as to the soundness of this procedure arose only once, 
when, in a discussion with the head masters of the senior secondary 
schools, the suggestion was made that it would be better to discard 
such subjects as Art, Handwork, Music and Physical Education, 
and take an average merely of the important and academically 
reputable subjects like English, Latin, French and Mathematics. 
It is, of course, quite true that from many points of view the 
subjects which matter most are those in which the pupil will aim 
at passes on the higher grade of the Leaving Certificate examina¬ 
tion, and which he will require for such purposes as university 
entrance. Yet we decided to take a broader and more generous 
view of secondary culture and to adhere to our principle of using 
the marks in all the subjects of the course. 

48 
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VVe were encouraged to persist in this view by the reflection 
that all the subjects would have fairly high inter-correlations, and 
by certain results obtained by Mr Robert Wilkie, B.Sc. He 
prepared two lists of average first-year success marks for four of 
the senior secondary schools, the first being based only on the 
‘important* subjects (Latin, French, German, English, History, 
Geography, Mathematics, Science), the second being based on all 
the subjects. The average correlation between the two lists was 
+ •985. 

Importance of Comparability of Success Marks 

This question of the comparability of the success marks is one 
of the most important in the whole technique of follow-up in¬ 
vestigations. Its difficulty may be illustrated by considering the 
June Qualifying pupils who proceeded to the junior secondary 
courses. They distributed themselves among different schools, 
different courses in the same school, and different classes in the 
same course. We could not expect that the standards of the 
follow-up marks in the different schools or courses would be the 
same; therefore we had to deal separately with each course in each 
school. This, as we shall see, raises serious statistical problems 
if the attainment levels of the pupils in the different schools and 
courses vary. Furthermore, there is the possibility that the 
standards of marking will differ even in the various classes in 
the^ same course in the same school. These difficulties can, we 
believe, be overcome. But far the soundest policy is to evade 
them by taking every possible step to ensure that the follow-up 
marks are comparable in the first instance. 

Steps taken to ensure Comparability of Success Marks 

This question was discussed at various meetings with the head 
teachers of the senior and junior secondary schools and we con¬ 
cluded that there was no feasible method of ensuring uniformity 
m standard of follow-up marks for different schools, or for the 
different courses in the junior secondary schools. All we could 
reasonably attempt was to ensure that the follow-up marks would 
be comparable for the different classes in the same course in each 
junior secondary school, and that the marks would be comparable 

in the subjects common to the various courses in each senior 
secondary school. 

This was done by an extension of a system already in general 
operation in many of the schools, whereby pupils in the different 
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classes of the same course all take the same terminal examination 
papers. To secure uniformity in the standard of marking many 
of the head teachers adopted the plan whereby each teacher, 
instead of correcting the papers of his own class, was assigned a 
certain number of questions in the papers of pupils in all the 
classes. From later inquiries Miss Young ascertained the exact 
procedure that had been used in each school and course, and it 
was then found that it had not always been possible to adopt this 
system. In certain schools, although uniform papers were set, 
each teacher corrected the scripts for his own class, the standards 
being then co-ordinated by the head of the department. 

The final position, in short, was that in most schools we had full 
assurance that the follow-up marks were comparable; in others 
we had a fair measure of assurance; but in a number of schools 
there was no guarantee that the same standard had been used. 


Check of Comparability of Follow-up Marks in the 

Various Classes 

In the circumstances it was imperative that we should make a 
complete check to find which sets of marks should be excluded 
from our further calculations, and we did this in the following 
way. If, for the purposes of illustration, we consider the pupils 
who elected a particular course in one of the junior s^onda^ 
schools, these would be arranged in, say, four classes. A, B, C and D. 
For these four classes we have the Qualifying examination means 
which are comparable. Now both Qualifying examination and 
follow-up marks are measures of scholastic attainment made at no 
great interval of time; and we may therefore assume that il tne 
follow-up marks were comparable the follow-up means would e 
in the same order as the Qualifying means and separated by 


similar intervals. . 

If this assumption were sound, and if we were given the follow 

up mean of Class A, we could calculate from the 

what the follow-up means of the other classes ought to be it tn 

follow-up marks were comparable.^ 

I This is done by a method similar to that used when we J 

teacher.' estimate, so that the mean and pupU. from 

from a school were the same as those of the Quahfymg m F F 

that school. The formal® are of the following form: 

bMf = iMp - ^(aMq - bMq) 

where aMf = mean follow-up mark for Class A, 

AMQ=mean Qualifying mark for Class A 
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The first purpose for which we used this method was to test the 
validity of the underlying assumption that the Qualifying means 
give a sound guide as to the relative positions of the follow-up 
means. This was possible because in many cases we knew that the 
follow-up means were comparable, and we could compare the 
calculated means with them. Data for a case of this kind are 
given below: 


Class 

F means (per cent.) 
Actual Calculated 

d 


A 

73-2 

73-2 

• # 


B 

699 

681 

+ 1-8 

3-3 

C 

68-5 

67-6 

+ -9 

2-9 

D 

667 

647 

+ 20 

2-8 

E 

57-3 

6i-6 

“4-3 

3-5 

F 

49-6 

SI-3 

-17 

31 


The figures reveal the width of the gap that separates the 
best from the poorest class, a point whose statistical import in 
follow-up work we shall discuss presently. They also show that 
the calculated means agree very closely with the actual means, 
the differences being in all cases explicable by the fallibility of the 
measures concerned.^ 

This method was applied to every class in every course in every 
school in all the three years of the follow-up, and the assumption 
stood the test in all cases where we had full assurance of the com¬ 
parability of the follow-up marks. We therefore felt confident in 
using the check as a criterion for discarding groups for lack of 
comparability. 

We give on p. 52 the data for a school in which we had not 
full assurance of comparability, but which we regarded as satisfy¬ 
ing the test. 

\ error which is relevant here does not concern the variations 

which arise through taking different samples of pupils from a total population; 

It relates to the fallibility of the measurements. Knowing the fallibility of the 

the calculated follow-up mean is based we can calculate 

lU fallibihty and the fallibility of the difference between it and the actual 
mean. 

Reference: D. Brunt, The Combination of Observations. Cambridge: 

Cambridge University Press, 1917. P. 48. 

The calculations are somewhat intricate and laborious, and in view of the 
number of cases with which we had to deal, and of the fact that great accuracy 
was neither possible nor necessary, the method used was to work out sample 
case# and mterpolate by means of a graph. 
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Class 

F means 

Actual 

(per cent.) 

Calculated 

d 

pe&m 

A 

64-6 

64-6 

• • 

• • 

B 

6i-3 

57-6 

+ 3-7 

27 

C 

54-1 

50-1 

+40 

2-6 

D 

50-9 

49-4 


3-0 


The following is a case where the test of comparability was not 
satisfied and the group of classes was therefore discarded from all 
further calculations: 


Class 

A 

F means (per cent.) 
Actual Calculated 
630 630 

d 

4 • 

pm. 

4 4 

B 

63-4 

53*6 

+ 9 *® 

2*4 

C 

58-3 

45-4 

+ 129 

2-4 


The amount of labour involved in this check was very great 
We felt, however, that we were repaid by the confidence with 
which we could proceed with the further analysis of our data. 

If we leave out the top classes, where the calculated mean was 
arbitrarily fixed to be the same as the actual mean, the result 
was that in 6g per cent, of the cases the differences between actual 
and calculated follow-up means were less than their probable 
errors; 96 per cent, of the differences were less than three times 
their probable errors. The number of rejections was therefore 
not very great. 

A point which emerged from these calculations was that there 
is a general tendency for the follow-up means of the duller classes 
to be a little higher than one would expect on the basis of perform¬ 
ance in the Qualifying examination. The difference is not large, 
the average value being 2 per cent., but it is almost universal. It 
may be due simply to greater leniency in marking the poorer 
classes, or to a tendency to push these classes nearer to the hmit 
of their capacity. 

Comparability of Follow-up Marks in the 

Various Schools 

As has already been mentioned, we did not expect ^at the 
follow-up marks would be comparable between sc^ols or between 
courses in the same junior secondary school. The latter was, 
indeed, no great disadvantage, for we wished to have separate 
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correlations for the various courses to see whether there were any 
significant differences. It may, however, be of some interest 
in connection with follow-up technique to show how the devia¬ 
tions in standard between school and school compare with those 
between class and class in the same school and course. We give 
below a comparison of actual and calculated follow-up means for 
the June group taking the boys’ technical course in junior secondary 

schools: 


School 

F means 

(per cent.) 

d 


Actual 

Calculated 


A 

539 

53-9 

• « 

• • 

B 

57-5 

58-0 

- '5 

1-4 

C 

58-5 

53-8 

+ 4-7 

1-4 

D 

43-3 

S 7 -I 

-13-8 

1-4 


Schools C and D differ in standard of marking by 18-5 per cent., 
and this holds almost exactly for all the courses in those two 
schools. In the commercial course, for instance, the difference is 
18 4 per cent. 

A comparison of actual and calculated follow-up means in senior 
secondary schools is shown below: 


School 

F means 

(per cent.) 

d 



Actual 

Calculated 


A 

64-8 

64-8 

• 4 

• 4 

B 

617 

693 

- 7-6 

1*1 

C 

649 

649 

00 

• 4 

D 

646 

669 

- 2-3 

i '3 


The position is, therefore, that, thanks to the co-operation of 
the head teachers and staffs of the secondary schools, we could 
regard the follow-up marks of the classes in the same course in the 
same school as comparable, but that the standards as between 
school and school, and as between the different courses in the 
junior secondary schools, could not be regarded as comparable. 



CHAPTER VII 

METHODS USED IN COMBINING CORRELATIONS 

This is a tiresome chapter, devoted to certain points of follow-up 
technique, which the general reader will be well advised to omit. 


The Problem 

Suppose we wish to find the best estimate of the correlation rgp, 
and that we have six schools for which the follow-up (F) marks 
are not comparable though comparable within each school. 
Suppose also that the ability levek in the schools are different. 

One method of finding an estimate of tqp would be to calculate 
a correlation for the combined schools from a single grid, but this 
would involve an error caused by the differing standards of the 
follow-up marks. Another method would be to calculate separate 
correlations for the six schools and combine these by the usual 
z transformation. This, while avoiding the error of the first 
method, would introduce another, due to the differences in the 
ability levels of the schools. We shall call the first the error 
arising from differences in standard of marking, and the second, 
that arising from differences of the means. 

The problem is to find a method of combining the correlations 
which avoids both sources of error. 


Size of Error caused by Different Standards of Marking 

We can get a general idea of the error arising from differences in 
standards of marking as follows. If we have two large schools of 
the same size whose follow-up marks are on the same standard, 
the correlation for the combined schools would be 

54 
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If the follow-up marks for one of the schools were all raised by 
a per cent., the formula would become 


na. 


j 


Thus a small variation in standard would not have a serious effect, 
but a difference of 15 per cent, might lower the correlation by as 

much as*i. 

The following correlations taken from Mr Bam s data give 
actual examples of the error in correlation caused by different 
standards of marking in one variate: 

English 
(N=858) 


Slumped correlation with 
teachers’ estimates 
Slumped correlation tvith 
teachers’ estimates scaled 

Loss in correlation . 


+ 766 
+ ■879 

•113 


/Vrithmetic 

English and 
Arithmetic 

(N= 853 ) 

(N=842) 

+ 727 

+ 787 

+ ■799 

+ •883 

•072 

•096 


The differences in standards of marking are shown in Tables 
XIII and XIV, and we see that they give rise to a loss of approxi¬ 
mately •! in correlation. 


Size of Error caused by Differences in the Means 

To illustrate the size of the error from the second source we take 
the six classes whose means are given on p. 51. In this case we 
know that the standards of marking are comparable, yet the 
difference of means is considerable, the average of the best class 
being 73*2 per cent, and that of the poorest 49*6 per cent. 

A simple diagram will show that there would be a considerable 
error if we used the average of the correlations. A correlation 
can be represented pictorially by an ellipse which is a horizontal 
section of the normal bivariate surface. Omitting certain technical 
reservations about scales and standard deviations, we may interpret 
a broad ellipse as a low correlation and a long narrow ellipse as a 
high correlation. From the data for the six classes we can find 
the centres and the shapes of the ellipses and the positions of their 
major axes, and plot these on a diagram. 

Fig. 6 shows the ellipses for the best and poorest classes. AB 
is the line of centres, and the other two lines are the respective 
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SUCCESS IN SECONDARY SCHOOL 
FOLLOW-UP 

Fig. 6.—Effect of differences in the means when the differences are 

nearly the same in both variates* 


--1B 

I 
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Effect of differences in the means when the difference in the 
v-variate is one-third of that in the x-variate. 
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major axes. The centres of the intermediate ellipses lie closely 
along AB. 

While a single class gives a broad ellipse corresponding to a low 
correlation, the whole group would be represented by a much 
longer ellipse corresponding to a considerably higher correlation. 

The correct correlation for the complete group, however, is not 
always higher than that for the homogeneous sub-groups; every¬ 
thing depends on the position of the means. If the difference 
in means of the y-variate were one-third of the difference for the 
ar-variate, the position would then be that shown in fig. 7. 

It is clear from this diagram that the correlations for the whole 
group may be the same as that for the sub-groups, or even lower. 

To find the size of the loss in the case of these six classes, we 
calculated the correlation from a single grid and also by averaging 
the correlations for the separate classes. The results are as follows: 

Average of correlations for six classes . + *404 

Correct correlation for whole group from 
single grid.-{-•619 

Loss through averaging .... ‘215 

The loss in correlation in this way is well known,^ but it is 

perhaps not generally realised that the difference could be as high 
as *215 

Methods of avoiding Error caused by Differences 

IN THE Means 

The above examples show that the error might be serious; and 
the question now arises whether it would have to be taken into 
account, say, in combining correlations for the senior secondary 
schools, where the differences in means would be smaller. What 
we require is a method which will show whether a correction is 
required, and will enable us to make it if necessary. 


USE OF A CORRECTION FOR RANGE ^ 

It might be thought that we could overcome the difficulty 
simply by correcting the correlations for the separate groups for 

A First Course in Statistics. Boston: 
Houghton Mifflin Company^ *938. P. 179. 

* C/- G- H. Thomson, The Factorial Analysis of Human Ability. London: 
T T London Press, Ltd., 1939. Chapter xi. 

1. L. Kelley, Stattsttcal Method. New York: The MacmUlan Company, 
I 923 * P.223. ^ 
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range, but consideration of the possible arrangements of the 
ellipses will show that this would often be unsound. Further, 
if there were lack of comparability in both variates it would be 
impossible to obtain a satisfactory estimate of the standard devia¬ 
tion of the total population. 

SCALING THE FOLLOW-UP MARKS 

By using a process of scaling on Qualifying examination marks 
similar to that described in Chapter V, we can make the follow-up 
marks comparable and then calculate the correlation for the 
combined schools from a single grid. In Chapter VI we have 
given grounds for our belief that the underlying assumptions are 
legitimate, and another point in confirmation will be made at the 
end of the next section. 

Instead of applying this method in the direct way, we found it 
better to use the following short-cut. 

CALCULATION OF FOLLOW-UP CORRELATIONS WITH SCALED 
FOLLOW-UP MARKS WITHOUT ACTUAL SCALING 

Since the scaling is done by equations, we can use these to find 
formulae giving the values of the correlations with the scaled 
follow-up marks. 

If rjF(Q) denotes the correlation between intelligence score and 
follow-up marks for all the schools. A, B, C, D, . . ., when the 
follow-up marks are scaled on Qualifying examination marks, 

Sma<^ia<^qa^ipa + _ 

" V{2nA<^iA+ + 

where rjpA = correlation between intelligence score and follow-up 

mark for School A, 

«A = frequency for School A, , 

ctqa = standard deviation of Qualifying marks for pupils in 

School A, r c v 1 A 

a = deviation of mean intelligence score for School A 

from mean intelligence score for all schools, 

*= deviation of mean Qualifying score for School A 

from mean Qualifying score for all schools, 

S denotes summation for all schools, A, B, C, D, . . . 

Formula ^ of this kind have some general interest. For instance, 

1 These formula: can readily be put into terms of the correlations between 
tht school means. 
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if we had calculated the correlations for the individual schools, we 
could find the true correlation for the combined schools. If the 
follow-up marks were known to be comparable, we could substitute 
(TpA. for (tqa, ‘tnd so on. The formulae also show at a glance the 
effect of combining correlations when the means are the same but 
the standard deviations different. In this case, <3 = ^ = . . .—p=^q 
= . . . =0. 

As a test of the formulae and of the assumptions on which the 
scaling system is based, we used one of them to calculate the true 
correlation for the six classes used as a previous illustration. From 
p. 57 it will be seen that the average correlation for the six classes 
is +*404 and that the true correlation is +- 6 i<^. With the 
formula the result is +*6i8. 


Application of Formula to determine Necessity for 
Correction for Difference in Means in Follow-up 
Correlations for Combined Schools 

On applying the formula we found, as we expected, that the 
differences between the average and the true correlations were 
small, being only about -007. Now, such a loss would be worth 
saving if we could rely upon our data to that degree of precision 
and if we were absolutely confident of the exactness of the method. 
Neither of these conditions for statistical tampering with results 
was, however, satisfied; and we decided, with reUef, that it was 
sufficient to use simply the average values. 


Application of Formula to determine Necessity for 
Correction for Difference in Means in Combining 
Correlations for June and December Groups 


At no point in our investigation did we wish to be in the position 

in error by some un- 

rase w>,‘" “ another 

case where we knew there was a difference of means. This arose 

DecembeT,n7T‘°“ Q^aUfying data correlations for the 

correlations between 

comra^^n this^case^there" was Iac?o 1 



6 o 


SELECTION FOR SECONDARY EDUCATION 


TABLE XX 


December and June Qualifying groups: Difference in intelligence 

and school attainment 



December group 
(N = 1,045) 

June 
CN = 

group 

L 9 +o) 

Mj-Md 

M 


M 

<7 

1 

IQ 

98-47 

12-53 

101-80 

14-50 

3-33 

EQ* 

98-22 

13-65 

101-67 

14-10 

3-45 

AQ* 

lOI-IO 

1510 

I02-8o 

H -03 

1-70 


* £Q denotes English quotient and AQ Arithmetic quotient. 


The differences in mean are statistically significant, and the 
question we have to answer is whether they are large enough to 
warrant the making of a correction in combining the correlations 
for the two groups. 

The fact that both variates have to be scaled offers no mathe¬ 
matical difficulty, provided that we have a suitable basis for the 
scaling. In this case the only possible bridges are IQ, EQ and AQ, 
and we scaled I on IQ, Qe and E on EQ, and Qa and A on AQ. 

If we call the correlation between English and Arithmetic, 
when English is scaled on EQ and Arithmetic on AQ, (d+j/e(bq)a(aq)> 
the formula is found to be as follows: 

Skd d^^aq d<^bq d^ea + 

(D+J)^E(EQ)A(AQ)= 

where ^ and y = deviations of the English means of the 

December and June groups from the mean 
of the combined group, 

/and / = deviations of the Arithmetic means of the 

December and June groups from the mean 
of the combined group. 

Formulae for other variates can easUy be written down once 
the principle of their construction has been grasped; and the 

results of their application are given in Table XXI. 

Again it appears that the error in our combined correlations 

arising from this cause is negligible. 
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TABLE XXI 

December and June Qualifying groups: Correction of combined 

correlations for differences in means 


Correlation 

Correction 

1 and £ 

+ 002 

I and Qe 

+ 002 

E and Qe 

+ •002 

A and Qa 

+ 001 

I and A 

-003 

I and Qa 

-•003 

E and A 

-•001 

E and Qa 

- 001 

Qe and Qa 

- 001 

Qe and A 

-001 


In more than half of the cases the correction is negative, showing 
that the method of averaging correlations sometimes gives an ov^r- 
estimate even when both means for one group are higher than the 
corresponding means for the other. The explanation of this is 

given on pp. 56-57. 


Variation of Correction with Size of Correlation 

AND of Difference in Means 


In the correlation Tiqb difference of means in both variates 
was about 3*4. quotient points, and we concluded that a correction 
was not necessary. It is of some interest to find whether it would 
have been necessary if the difference had been, say, double the 
amount, and also how the correction would have been affected 
if the correlation had been lower. The size of correction for 
different sizes of correlation and of differences in means is shown 
below; 


Size of correlation 

Size of correction when differ 
in means is 

3'4 6*8 

+ •844 

+ •002 

+ •008 

+ •600 

+ •004 

+ •020 

+ •300 

+ •009 

+•036 

■000 

+ 013 

+•052 
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These results are not of general appUcation, yet they show that 
the correction is larger for the smaUer correlations and that its 
size rises steeply when the difference of means is increased. 

Special Difficulty in Correcting for Differences in 
Means in Follow-up Investigations 

In follow-up correlations we have seldom to deal with lack of 
comparability in the first variate. Our problem lies in the follow- 
up marks, and we have suggested that these should be scaled on the 
Qualifying examination. In doing so we are making the differences 
in mean the same for both variates in Tqf. It is, of course, very 
probable that they ought to be the same, but we are giving the 
benefit of the doubt to this one correlation, and thus putting it in 
a privileged position when we compare it with others. 

It was partly for this reason that we decided to make no correc¬ 
tion in our own follow-up, where all the corrections were small. 

Conclusions in regard to Follow-up Technique 

Our experience of this problem points to the following con¬ 
clusions in regard to follow-up technique; 

Every possible step should be taken to ensure that the follow-up 
marks of the classes are comparable within each school. If 
they are not comparable they may be scaled on Qualifying 
examination marks. 

As far as the foUow-up marks are concerned, it will normally 
be permissible to combine the correlations for the separate 
secondary schools by the 2 transformation, but before doing 
so the size of the correction for differences in means should 
be ascertained by the methods described above. 

The experiment should be so planned that scaling of the first 
variate is seldom necessary; and such scaling should be done 
only on a bridge in the same subject. 

Check that all Correlations which were combined could 
be regarded as derived from Equally Correlated 
Populations 

This was done by the method.^ The total number of 
correlations involved was over 200, but it was usually obvious that 
the condition would be satisfied. The statistical check was 

^ Cf. P. R. Rider, An Introduction to Modern Statistical Methods^ p. 106. 
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applied in doubtful cases, and only in four of these did P lie below 
•01. Fortunately they fell in groups which were rejected on 
account of lack of comparability of follow-up marks, or concerned 
correlations with teachers’ raw estimates. 


Variance and Co-variance within and among Schools 

The statistical reader will have noted the connection between 
the above formulae and the methods of analysis of variance and 
co-variance, and it is possible that these methods could prohtably 
be used in the study of the problems of this chapter. The variance 
for the five senior secondary schools is analysed below, the data 
used being the Qualifying examination marks: 



Degrees of 

Sum of 

Mean square 


freedom 

sqUiires 

deviation 

Within schools . 

364 

70,466 

*94 

Among schools. 

4 

105 

26 

Total . 

. 368 

70.5 7 > 



In this case z — h log* w = i *00. 

The 5 per cent, value of z for = 24 is ‘877, and the i per cent, 
value is 1*317. For n^ — 00, the 5 and i per cent, values are -864 
and 1*300. The obtained value for the significance of the variance 
among schools is therefore between the 5 and i per cent, points. 

The largest variation among schools in all our follow-up data 
was that in the commercial course for the December group, and 
it was only after some hesitation that we decided to make no 
correction for differences of means even in this case. It was, 
however, a small group, and we felt that the absence of correction 
would make little difference in the final results. Below we give 
the analysis of variance and co-variance for this group of four 
junior secondary schools, the data being for the correlation rj^: 


Degrees of Sum of 
freedom squares 

A 

Within schools . 120 13>277 

Among schools . 3 i ,67^ 

Total . 123 I 4 » 95 i 


Sum of 
products 

5.824 

712 

6,536 


Sum of 
squares 

F 

10,847 

804 

11,651 


This shows a much greater variation among schools, but even 
here the regression among schools is not significant. 



CHAPTER VIII 

COMBINATION OF CORRELATIONS 


The senior secondary group was distributed over five schools, 
while the main body of the junior secondary pupils went to four 
large schools, where they split up into three courses. 

This complication of groups entailed the calculation of approxi¬ 
mately 1,000 follow-up correlations, which have a somewhat 
oppressive effect if presented in the mass. We therefore give 
merely sample Tables to show the stages by which we arrive at the 
final combined results. 


SENIOR SECONDARY SCHOOLS 
Original Tables for Separate Schools and Years 

TABLE XXII 


Sample Table of correlations for a senior secondary school 


Correlation 

with 

iFi 

iF, 

iF, 

iFat 

iFat 

.Fat 

I 

715 

• 7 H 

■680 

•710 

■653 

■639 

IQ 

•696 

•694 

•703 

■707 

•599 

•610 

E 

•580 

•616 

•586 

•607 

•581 

•515 

A 

700 

•697 

•698 

•726 

•675 

•597 

Qe 

• 7+3 

732 

■731 

•760 

•636 

•584 

Qa 

•662 

•669 

1 .69s 

•710 

•665 

•616 

S 

707 

725 

709 

•726 

-694 

•614 

Q 

•773 

771 

.785 

•809 

•716 

•660 

T 

716 

•68s 

-679 

•688 

•632 

.572 

Ts 

• • 

• • 

• • 

•774 

•654 

•652 


Even in this single school the correlations are reasonably steady, 
and many of the final results cast their shadows before. For 

64 
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instance, Ts always shows a higher correlation than T; the correla¬ 
tions for Qe are always higher than those for E. This fact, that 
most of our results held not merely in the final combined Tables 
but for every school and for every year, added greatly to our 
confidence in them. 

Only a very small number of the entrants to the senior secondary 
schools were from the December group, and these were omitted. 

First Condensation: Combination of Schools 

We had five Tables of the above type, and the next step was to 
reduce them to a single Table by combining the correlations for 
the five schools. 

TABLE XXIII 

Correlations for senior secondary schools combined 



Correction of Second- and Third-Year Correlations 

FOR Range 

The differences between the correlations for the three terms of 
the farst year are not significant statistically, so we may conclude 
that, on the average, the results do not estabhsh a faU during the 


3 
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The slight rise from and jFg to iFat might also be due 
to the fluctuations that are to be expected with relatively small 
numbers. There is, however, a mathematical explanation, in that 
the correlation with an average will usually be less than the 
average of the correlations with the elements which make up 
the average.t 

We can also explain part of the fall from year to year. There 
is little or no leakage in the first year, but thereafter the numbers 
fall rapidly, due largely to the dropping out of weaker pupils. 
The narrowing of the range of talent, indicated by the standard 
deviations at the foot of Table XXIII, will cause a fall in the 
correlations. 

We can correct for this narrowing of range, ^ and the correlations 
then approximate to what they would have been if all the pupils 
had gone on to the end of the third year. This has two advantages: 
it makes the correlations for the various years comparable, thus 
removing one obstacle to their further combination; and it 
enables us to show the true fall in predictive value from year to 
year. (See Table XXIV.) 


Final Condensation: Combination of Correlations 

FOR THE Three Years 

The main aim of this chapter is to show the successive stages 
by which the correlations were condensed to single values for 
each measure, and further discussion of the fall from year to 
year is deferred until the corresponding results for the junior 
secondary schools are available. Until then, we shall also provision¬ 
ally regard the other conditions for the combination of the correla¬ 
tions for the different years as satisfled. (See Table XXV.) 

^ The average of the correlations with I is 

riy,+rif,+rip, 

3 

The correlation with the average is 


if Ft. F, and F. are given equal weights. j .u 4 

S!nc=%...., r,.,. and r,... wiU norcaUy be les, than the tecond method 

tvAuld 2 higher result tbdn the first- j j ^ 

« c/p C 7 . The narrowing of the range in follow-up marks was regarded as 

consequentUl upon the narrowing of the range m the first vanate. 
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TABLE XXIV 

Correlations for senior secondary schools combined: Second- and 

third-year results corrected for range 



TABLE XXV 

Final order of predictive value of measures for success in 

senior secondary schools 
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Correlations for Batteries of the Original Measures 

Few Education Committees would now, we hope, use a single 
test or examination for purposes of guidance or selection. They 
would be more likely to use a pair, like IQ + Q, or a battery of 
three, like IQ + Q+Ts. We have now, therefore, to find the 
order of merit for all the possible batteries,^ and for the reasons 
offered in Chapter X we gave all measures equal weights. 

The final correlations for batteries are given in Table XXXII, 
along with those for the junior secondary schools. 


JUNIOR SECONDARY SCHOOLS 


The results for the junior secondary schools are much more 
complicated and the number of original correlations much larger. 


Original Tables for Separate Schools and Years 


TABLE XXVI 


Sample Table of correlations for a junior secondary school 

(Boys’ technical course) 


Correlation 

with 

1 

December group 

June group 

iFat 

.Fa. 

iFa. 

*Fat 

I 

•6o6 

•610 

•670 

■718 

IQ 

.569 

754 

•650 

•673 

E 

•419 

.415 

.536 

•534 

A 

•627 

•664 

•548 

•529 

Qe 

•632 

•609 

.709 

•610 

Qa 

.548 

•596 

•545 

•520 

T 

•641 

•630 

•517 

•554 

Ts 

•708 

•697 

•806 

•’JZ% 


There were twenty such Tables. 

1 No reference, need be given on the nnethod. u«d here; they will be found 


in any text-book of statistics* 

The following is a sample formula: 

rj p fqp +^8F _ 

r(i+Q+8+3B)F = + 2riQ + 2ria + Zfm + ^''Qs + 

All the inter-correlations of the measures are therefore necessary, and these ha 
fo be obuined separately for the senior and jumor secondary groups. 






TABLE XXVII 

Correlations for junior secondary schools combined 


COMBINATION OF CORRELATIONS 69 
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We had four to six schools, three courses, and December and June 
groups. Only for the commercial course was it possible to obtain 
correlations for aU three years of the follow-up; for the boys’ and 
girls technical courses the numbers in the third year were too 
small. (See Table XXVI, p. 68.) 

. First Condensation: Combination of Schools 

The fipt step is to combine the schools after ascertaining that 
it is statistically legitimate to do so. When this is done we get 
the results shown in Table XXVII (p. 69). 


Second Condensation: Combination of December and 

June Groups 

The December and June groups can be regarded as equally 
correlated populations, and we therefore combine them, giving 
the results shown in Table XXVIII. 

TABLE XXVIII 


Correlations for junior secondary schools: Schools combined: 
December and June groups combined 


Correlation 

with 

Boys’ 

technical 

course 

Girls* 

technical 

course 

Commercial 

course 


iFav 

.Fa. 

iFav 

.Fa. 

iFa. 

.Fa. 

.Fa. 

I 

•616 

•589 

■631 

•583 

.529 

•519 

■568 

IQ 

•595 

•595 

•609 

•624 

•520 

•502 

•557 

E 1 

1 

•523 

•478 

-509 

•516 

•461 

•428 

•457 

A 

•468 

■410 

•573 

•521 

.511 

• 5*7 

•498 

Qe ! 

•632 

•597 

•604 

•562 

•595 

•555 

■561 

Qa 

•501 

•475 

•531 

•418 

.491 

•410 

•468 

S 

•628 

•561 , 

•633 

•^5 

.581 

•566 

•579 

Q 

♦667 

•631 

•661 

•570 

■645 

•609 

•606 

T 

•578 

•593 

•604 

•528 

•531 

•536 

•570 

Ts 

•661 

.649 

•640 

•611 

•640 

•627 

•599 

Average r 

-587 

00 

• 

•599 

' -554 

.550 

■527 

.546 

N 

768 

533 

541 

333 

342 

294 

146 
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Correction of Second- and Third-Year Correlations 

FOR Range 

The following correlations are those that would have been 
obtained if the whole of the first-year group had completed the 
course: 


TABLE XXIX 


Correlations for junior secondary schools: Schools combined: December 
and June groups combined: Second- and third-year correlations 
corrected for range 


Correlation 

with 

Bovs’ 

technical 

course 

Girls’ 

technical 

course 

Commercial 

course 

iFav 


iPat 

jFat 

iPav 

fPAv 

sPav 

I 

' -616 

■621 

•631 

•606 

■529 

■ 5>5 

■550 

IQ 

•595 

■627 

•609 

■647 

•520 

•498 

■538 

F. 

•523 

•509 

•509 

•539 

■461 

•423 

•434 

A 

•468 

•439 

•573 

•544 

•5II 

• 5>3 

•477 

Qe 

•632 

•629 

•^04 

•585 

•595 

• 55 ' 

•542 

Qa 

•501 

•506 

• 53 « 

•^39 

•491 

•405 

•445 

S 

•628 

•593 

•633 

•628 

•581 

■563 

•562 

Q 

•667 

•662 

•661 

•593 

■645 

•607 

■ 59 ' 

T 

•578 

•625 

•604 

•551 

■ 53 * 

•532 

•552 

Ts 

•661 

•680 

•640 

■634 

•640 

•625 

•583 

Average r 

■587 

■589 

•599 

•577 

■550 

•523 

■527 

Average r 
uncorrected 
for range 

■587 

-558 

•599 

■554 

•550 

•527 

■546 

N 

768 

533 

54 » 

333 

342 

294 

146 
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Third Condensation: Combination of Correlations 

FOR Different Years 

TABLE XXX 


Correlations for junior secondary schools: Schools combined: 
December and June groups combined: Years combined 


Correlation with 

Boys* technical 
course 

Girls* technical 
course 

Commercial 

course 

I 

•618 

•622 

•528 

IQ 

•608 

•624 

■ 5*5 

E 

■517 

•521 

•442 

A 

•456 

•562 

•506 

Qe 

■631 

•597 

■569 

Qa 

•503 

•497 

• 45 * 

S 

'614 

•631 

• 57 * 

Q 

•665 

•636 

•621 

T 

■598 

•585 

•535 

Ts 

•669 

•638 

•624 


Fourth Condensation: Combination of Correlations 

FOR Different Courses 

TABLE XXXI 


Final order of predictive value of measures for success in 

junior secondary schools 


Order of merit 

Final combined correlation 
with success 

Qe 

•605 

A 

•502 

E 

•499 

Qa 

-487 

Ts 

•648 

Q 

•645 

S 

•608 

I 

■597 

IQ 

.590 

T 

•578 
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We shall see in the next chapter that the statistical requirements 
for this combination are satisfied; but even if they were not, it 
would still have been desirable to make it, for only thus can we 
arrive at a final general order of merit of predictive value applicable 
to all post-primary courses. (See Table XXXI.) 


Simplification of Tables for Batteries by Omission of 
Correlations with Intelligence Score and Teachers’ 
Unscaled Estimates 

Our next step is to find the correlations for batteries; but, in 
view of the results already given, there is no point in retaining 
data for the teachers’ unsealed estimates, which should not be 
used for prognostic purposes. 

After making this simplification we are still left with 23 different 
combinations of the fundamental measures, none of which includes 
both intelligence quotient and intelligence score. If we could 
justify the omission of either IQ or I we could reduce the number 
to 15. 


comparative predictive value of intelligence score and 

intelligence quotient 

The correlations for all the combinations, one of which includes 
intelligence score and the other intelligence quotient, were cal¬ 
culated, and the differences are shown below: 


Battery 


I . 

I + Q . 

I + S . 
I+Ts . 
I+Q + S 
I + Q+Ts 
I + S +Ts 
I + Q + S+Ts 


Correlation for battery witl; I, minus 
correlation for battery with IQ 


Senior secondary 
schools 

+ 009 
+ 014 
+ 008 
- 003 
+ 009 
+ 004 
+ 001 
+ 004 


Junior secondary 
schools 

+ 007 
+ 002 
•000 

- 003 
•000 

- 001 

“ -003 

-001 


The average difference is -004; none of the differences is 
Significant statistically; I is higher in nine cases> IQ in five cases, 
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and in two cases the correlations are equal. Hence, so far as our 
data go, the two may be taken as equal in predictive value. 

This point has a bearing upon the question of age-allowances, 
and also upon the difficulty created by the limited supply of tests 
with reliable norms. If we have only three or four from which 
to choose, we cannot avoid repetition and the possibility of candi¬ 
dates preparing for all of them. Since, however, intelligence score 
is as good as intelligence quotient, we may use an unstandardised 
test and thus widen our range of choice. Of course we could not 
then put the results in the familiar and easily interpreted form of 
IQ; but we feel that, in view of the lack of comparability between 
IQs determined by different group tests, this might be an ad¬ 
vantage rather than a disadvantage. 

With some hesitation we decided to omit intelligence score 
rather than intelligence quotient. This is not so sound statistically, 
for we mix scores and quotients; but the use of IQ is more in 
conformity with current practice. 


Correlations for Batteries of the Original Measures 

The correlations for the junior secondary schools are lower than 
those for the senior secondary schools. Now we should expect 
that the range of talent in the intellectual Hite that enters the 
senior secondary schools would be narrower than that in the 
junior secondary courses; and that would no doubt be so to-day. 
In 1935-1936, with no strict selection, the standard deviation of 
the Qualifying examination marks for the senior secondary group 
was considerably larger than that for the junior secondary groups. 
We therefore applied a correction for range to the correlations 
for the latter, thus permitting the comparison shown in Table 

XXXII. 

The agreement is remarkably close, both in size of correlation 
and in order of merit. It is, of course, not necessary that this 
should be so; and the differences are not necessarily due to 
sampling or experimental errors. It is quite possible that the 
order of predictive value may be different for the two types of 
school. 

We started with nearly 1,000 correlations. These have novr 
been condensed to 8, those for IQ, Q, S and Ts, and then expande 
to 30. In performing this condensation we may, at times, have 
taken a little statistical licence, but Table XXXII is worthy of 
respect in consideration of the labour involved in its preparation. 
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TABLE XXXII 


Predictive value of batteries for senior and junior secondary schools 


Battery 

1 

Senior secondary 
schools 

Junior secondary 
schools • 

d 

IQ+Q+Ts . 

•804 

-806 

- 002 

IQ+Q + S+Ts . 

•800 

799 

+ 001 

Q +S +Ts . 

•790 

784 

+ 006 

IQ + Q. 

•786 

774 

+ 012 

Q +Ts . . 

783 

•780 

+ 003 

IQ + Q + S . 

783 

773 

+ CIO 

IQ + S +Ts . 

-782 

793 

- 011 

IQ+Ts 

779 

795 i 

-- 016 

Q + S • • . 

77 + 

753 

+ 02! 

Q . . , 

770 

738 

+ 032 

S +Ts, 

764 

775 

- 01 I 

IQ + S . , 

736 

74 * 

- 005 

Ts . . . 

•720 

741 

- 021 

S ... 

•698 

705 

- 007 

IQ . . . 

•691 

•689 

+ 002 


Corrected for range. 












CHAPTER IX 


PRELIMINARY DISCUSSION OF THE FOLLOW-UP 

CORRELATIONS 

In this chapter we draw attention to certain points of general 
interest arising from the results just presented, and justify certain 
of the combinations we have made. 

Is THERE A Fall in the Predictive Value of the Qualifying 
Measures in the Later Years of the Secondary 
Courses? 

In the preceding chapter we have shown that there is a fall, 
but that this can be partly accounted for by the progressive 
narrowing of the ranges of the groups. We can now present in 
one Table all the data bearing on this question: 

TABLE XXXIII 


Average follow-up correlations for the first three years of the 

secondary courses • 


Course 

N 


r 


Year i 

1 

2 

3 

Senior secondary 

0^ 

00 

•704 

•661 

•687 

Junior secondary: 

Boys’ technical 

768 

•587 

.589 

«• 

Girls’ technical 

541 

•599 

•577 

•« 

Commercial. 

342 

•550 

1 

•523 

•527 


• The correlations for the second and third years have been corrected for range. 

When we apply the statistical tests * we find that none of the 
differences is statistically significant. 

» For the senior secondary and commercial courses the z* *®/.^** 
convenient. For the senior secondary courses, where the differences arc 
greatest, x* i® ^ therefore about -5. For the girls technical 

course the z difference is -034: $£=-019. 

76 
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While the differences in average correlations are not significant, 
it is possible that those for the individual measures might be. 
This was tested for the four main measures, IQ, Q, S and Ts, and 
it was found that, on the whole, the correlations might be reason¬ 
ably regarded as coming from equally correlated populations.^ 
These results give some statistical justification for the combina¬ 
tion of the correlations for the different years; and, while they do 
not rule out the possibility of a real fall from the first to the third 
year, they show that it is unlihely to be much in excess of '02. 
The point is of great importance, for if the fall had been consider¬ 
able, we should have had to recast all our ideas as to the possibility 
of selection at the Qualifying stage. Measures which give a good 
prediction of success in the first year only would be of little value. 


Rise in Predictive Value of IQ in Later Years 

OF Courses 

An examination of Tables XXIV and XXIX gives little or no 
evidence of change in the relative position of I and IQ in the later 
years of the courses; but it does confirm the view that intelligence 
will tell in the end, and that the IQ rises in predictive value 
relative to the other measures as the years go on. In every course 
the correlation of IQ with F rises from the first to the last year, 
while the correlations for S and Q always fall. The differences 
are not great, but their consistency adds to our confidence in their 
existence, A comparison of correlations for the first and last years 
for all secondary courses is given below: 


Measure 

IQ 

Ts 

S 

Q 


Average rise or fall from 
first to last ye.ir 

Rise 029 
Fall -016 
Fall 020 
Fall -047 


The puzzling fact that the correlations are higher for the third 
than for the second year may be due to chance fluctuations or to 
correction for range. There may, however, be other reasons. 
1 he pupils may work more consistently up to their capacity, and 


for the senior secondary schools is 
that of Q, where z*=7-3i and P hes between -02 and -05. In the junior secondarv 

difference which approaches significance is thit for Q in^he 

gull technical course. Here the as difference is .112: SE=.o6i 
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there may be increased steadiness in the teachers* standards of 
marking as the Leaving examination approaches. Again, the 
pupils who have left comprise for the most part those not in¬ 
terested in the secondary course, and in this group the correlation 
between Qualifying attainments and success would probably be 
low. 

This tendency for the third-year correlations to be higher than 
those for the second makes it difficult to give an exact statement as 
to the significance of the rise in the follow-up correlations with 
IQ relative to those for the other measures, as in some cases the 
second is the last year, and in others the third is the last. If we 
consider Q, the fall from the first to the third year in the senior 
secondary group is just significant; ^ and while the rise for IQ 
is not in itself significant, the difference in relative position of IQ 
and Q in the last year is decidedly so. We may therefore take it 
as very probable that in the later years of the course IQ rises in 
predictive value relative to the other Qualifying measures. 

Does the Order of Predictive Value change in Later 

Years of the Secondary Course? 

TABLE XXXIV 


Changes in order of predictive value of the measures in the 

first three years 



Place in order of predictive value 



Junior secondary 

Measure 

Senior secondary 

Boys* 

Girls’ 

Commercial 



technical 

technical 

course 



course 

course 



Year 1 2 3 

1 

i 2 

I z 

I 2 3 

Q 

I i I 

I 2 

1 4 

I 2 I 

Ts 

222 

2 I 

2 2 

2 1 ^ 

S 

3 3 4 

3 4 

3 3 

3 3 3 

IQ 

4 4 3 

4 3 

4 I 

4 4 4 


* Cf. footnote, p. 77. 
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On the whole the measures retain their places with reasonable 
consistency, especially when it is remembered that a change in 
order may be due to a very small change in the relative sizes of 
the correlations which determine it. As we should expect, IQ 
shows a faint tendency to improve its place in the later years. 

The answer to the question therefore appears to be that there 
is no clear change in the order, but that there is a change in the 
spacing. 

Are the Correlations Higher for some Courses than 

FOR Others? 

From the figures given in Table XXXIII it would seem that 
the correlations are highest for the senior secondary courses, very 
nearly equal for the boys* and girls’ technical courses, and lowest 
for the commercial course. But when we take account of the 
relative ranges, we find that there is no evidence of any difference 
between the senior secondary, the boys’ technical and the girls’ 
technical courses. There is, however, a statistically significant 
difference between these and the commercial course,^ and once 
again our confidence in this curious result is increased by the fact 
that it appears not only in the averages, but separately for the 
December and June groups, and also for each separate year. 

Does the Order of Predictive Value vary from Course 

to Course? 

TABLE XXXV 


Order of predictive value of the measures for the different courses 


Measure 

Senior 

secondary 

Junior secondary 

Boys’ 

technical 

Girls’ 

technical 

Commercial 

Ts 

2 

1 

I 

1 

Q 

1 

2 

2 

2 

S 

3 

3 

3 

3 

IQ 

+ 

4 

4 

4 


* If we take the first-year correlations for the girls’ technical and the com¬ 
mercial courses the s difference of the average correlations is -073: $£ = *023. 
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Our answer is, therefore, that the predictive value of the 
Qualifying measures is the same for the senior secondary, the boys* 
and girls’ technical courses, but slightly lower for the commercial 
course. 

The order is even more consistent than for the different years 
of the same course, and the only notable point is that Q takes first 
place in the senior secondary course, while Ts gives the highest 
correlation for all the junior secondary courses. If we take the 
separate senior secondary schools and years we have fifteen com¬ 
parisons between Ts and Q, and in all but three of these Q takes 
first place; in all but four of fourteen comparisons for the junior 
secondary schools Ts takes first place. 

The answer to our question is, therefore, that there is a prob¬ 
ability that Q is better for predicting success in the senior secondary 
schools than Ts, but that otherwise the order of the measures is 
the same in all types of secondary course. 



CHAPTER X 


THE WEIGHTING OF MEASURES 


Nominal and Actual Weights 


We first warn the general reader against two sources of misunder¬ 
standing: 

If Qe, with a possible mark of 200, and Qa, with a possible 
mark of 100, are combined to give Q; or 
If the possible marks in Qe and Qa are 100, and if, before 
combining, we double the English marks; 

in neither case are we necessarily giving English twice the weight 
of Arithmetic. 

The reason is, that the actual weights depend upon the relative 
dispersions of the marks combined. If and cta are the standard 
deviations of the English and Arithmetic marks, and if, before 
combining, we multiply the marks by the nominal weights, Wb 
and Wa (giving the combined score W^E + W^A), then the actual 
weights given are Wectb and W^o-a. 

When we combined Qe and Qa to give Q, the weighting was not 

: I, but a complex one depending upon the standard deviations, 
and specific to our own examinations. It was for this reason that, 
in finding correlations for the batteries, we used the simple standard 
system of equal weights. 

We must now consider the questions, whether a better pre¬ 
diction would be obtained by using a different weighting system, 
and whether there are any general rules in regard to weighting 
which Education Committees should follow. For this purpose 
we use a statistical technique ' which enables us to find the weights 
giving the highest correlation with F, called the best weightings 
and the size of the correlation for the best weighted battery. 


Weighting of English and Arithmetic 

The problem is to find what weights should be given to English 
and Arithmetic in combining Qe and Qa to give Q, or in combining 

Corr^»r^' ■‘‘ 9 'L Computation of Regression Equations, Partial 

Correlations,etc., BnttsbJournalojPsycbologysMoX.xxms^zn I,July 1932,p.64. 

Si 
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E and A to give S. To solve it Dr McIntosh and Miss Young 
calculated the best weighted correlations for the combined senior 
secondary schools and found the differences between these and 
the equal weighted correlations. The improvement effected is 
shown below: 


Com¬ 

bination 


Difference between best 
weighted and equal weighted 
correlations with F 



iFi 


1F3 

iFat 

sFav 

Qe + Qa 

•005 

•005 

•001 

•002 

•003 

E +A 

•007 

•006 

•007 

•007 

•009 


Best weighting 


1^1 iFg iF, 

3:2 3:2 I ; I 
2:3 2:3 2:3 


iFat 

3:2 2:3 
2:3 1:2 


The differences are, of course, all in favour of the best weighted 
correlations, and they show that the improvement resulting from 
best weighting is very small. When we take this in conjunction 
with the facts that the best weightings are different for the tests 
and for the examinations, that they vary even for the same pair of 
examinations, and that the best weightings given above are specific 
to our particular tests and examinations and would not necessarily 
hold for others, we may conclude that the weighting of English 
and Arithmetic in Qualifying tests or examinations need not be 
attempted. 

Batteries—Best Weighting 


The improvement effected by the best weighting of batteries, 
as shown by the increase in the follow-up correlations, is as 
follows: 


Senior secondary Junior secondary 


Battery 

schools 

Improvement Weighting 

schools ‘ 

Improvement Weighting 

I+Q+Ts . 

•004 

5^7:3 

•002 

6:5:8 

I +Q + S+TS 

•008 

6 : 9 : I : 4 

•010 

9 :1 ; 8 : 3 

I +Q . 

•005 

4:7 

•003 

2:3 

I +Q + S 

•013 

6 : 11 : I 

•007 

8 : 3 :12 

Q + S +Ts 

•004 

7:3:4 

•003 

3:4:6 

II : 5 : 16 

I +S +Ts 

•003 

5:4:7 

•009 

Q + Ts . 

•004 

4:3 

•000 

I : I 

I +Ts . 

•000 

5:6 

•002 

5 27 

Q + S . 

•008 

3:2 

•003 

I : 2 

S+Ts . 

•OOI 

9:11 

•002 

2 : 3 

I +S . 

•000 

1 : I 

•000 

4:5 


1 The calculations for the junior secondary schools were made by 
Miss M. Mitchell, M.A. 
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The improvement is very small. It would be well worth making 
if we could assign systems of best weights for various batteries 
which would be generally applicable, but the above results show 
that this is impossible; even with the same tests there are striking 
variations between the senior and junior secondary groups. 

Mr A. S. Robertson, M.A., using a group of four senior secondary 
schools, found the following variations in best weighting for the 
same tests and the same group of pupils: 


Best weightings for correlation with 


Battery 

iFx 

iF. 

1F3 

iFat 

.Fa. 

IQ + Qe + Qa 

5:12:7 


4:11:9 

4:12:9 

6:9:13 

IQ + E + A 

9:4:9 

6:5:10 

8:4:9 

8 : 4 : 10 

4:5:11 

Qe + Qa+T 

13:6:5 

13:7:3 

11:8:5 

12:7:5 

9:12:2 

E +A+T 

9:10:14 

9:11 :12 

9:10:14 

9:10:15 

8:11 :11 

IQ +Qe +Qa +T 

2;4 : 2 : 2 

1:5:3:! 

1 14:3 :2 

I : 4 : 3:2 

I : 3 : 5:8 

IQ + E+A+T . 

3 : 2 : 3 ; 4 

2:2:3:4 

3:2:3:+ 

2 : 2 : 3 : 5 

1:2:4:+ 


Some of Mr Kidd’s results show that the best weightings for a 
given battery vary considerably from school to school: 


Best weighting for correlation of 


School 

battery 

I + Q with 



iFi >F. 

iF. 

l^AT 

A 

2:3 1:2 

1 :4 

I : 2 

B 

1:3 2:5 

2 : 5 

I : 8 

C 

2:5 2:3 

3 : + 

2:5 


In view of these results we find it impossible to lay down a single 
principle even of the most general character. Since, in addition, 
the gain by any system of best weighting is very small, we recom¬ 
mend that Education Committees should simply take an ordinary 
average of the results of the tests which they employ. The time 
and labour involved in the calculation of averages with complicated 
weights would be better devoted to the scaling of the teachers* 
marks. 



CHAPTER XI 

CORRELATIONS WITH THE BEST BATTERY 

When no follow-up data are available as a criterion, a rough idea 
of the relative values of the various measures can be obtained by 
assuming that the battery of all the tests is liJcely to be the best, 
and calculating correlations with it. 

The method has too much of the character of a vicious circle 
to command great confidence, but since we have a follow-up 

TABLE XXXVI 


Correlations of batteries with best battery IQ + Q+Ts 


Battery 

(0 

Correlation 
with best 
battery 
IQ+Q + Ts 

(2) 

Correlation 
with best 
battery 
(on scale of 
follow-up 
correlations) 

( 3 ) 

Correlation 
with F 

( 4 ) 

d 

( 5 ) 

IQ + Q + S+Ts 

.996 

■799 

•799 

•000 

Q + S + Ts . 

•987 

•787 

•787 

•000 

IQ+S + Ts . 

-987 

■787 

•787 

•000 

IQ+Ts. 

.992 

■794 

•787 

+ •007 

Q +Ts. 

•980 

•777 

•781 

- 004 

IQ + Q . 

•985 

■784 

•780 

+ •004 

IQ+Q + S . 

•98+ 

•782 

■778 

+ 004 

S +Ts . 

•974 

•768 

.769 

-001 

Q +S . 

■979 

•775 

■763 

+ 012 

Q 

.970 

•762 

•754 

+ •008 

IQ + S , 

•955 

•740 

•738 

+ •002 

Ts 

■944 

■724 

•730 

-*•006 

S 

•938 

•715 

•701 

+ •014 

IQ 

•921 

•690 

-690 

•000 
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criterion our data allow us to make a test of its validity. For this 
purpose we used the best battery, IQ+Q+Ts, rather than the 
complete batter)'.^ 

Our interest lies in the way in which the order so obtained 
compares with that of the follow-up correlations. The latter, 
which are given in column (4) of Table XXXVI, are the average 
correlations for senior and junior secondary schools; and to make 
the comparison easier we have reduced those with the best 
battery to the same scale. This was done roughly by equating 
the highest and lowest values and placing the intermediate points 
by means of a straight-line graph. 

The conformity between the two orders is remarkable; it is, in 
fact, much closer than that between the follow-up correlations 
for the senior and junior secondary schools. The average 
difference, without regard to sign, is *004, as compared with 
■on for the differences in Table XJ^II. 

These results give some justification for the belief that the 
method has evidential value; and, in view of the labour involved 
in large follow-up experiments and the time that must elapse 
before results are available, all ways of making legitimate short¬ 
cuts are important. Methods of this kind have, of course, serious 
limitations. If a new battery were prepared we could find its 
correlation with IQ + Q-i-Ts. If this came out to be -996 we 
could say that it was probably a very good battery, but we could 
not say whether it would give a higher or lower correlation with 
success. If, on the other hand, the correlation were *4, it would 
probably be a poor battery. 

^ The correlations were not calculated directly from grids but by means of 
formuls of the type mentioned in the footnote on p. 68. 



CHAPTER XII 


PROFILES: A SECOND METHOD OF DETERMINING 
THE PREDICTIVE VALUE OF MEASURES 

Each child at the Qualifying stage sets a problem: by the end of a 
follow-up inquiry we know the correct answer. We can therefore 
study any suggested procedure of selection, using as our test the 
number of cases in which our answers would have been wrong. 


Limitations of the Correlation Technique 

The correlation technique, which we have used so far, has many 
advantages, but it involves an immense amount of labour and 
has serious limitations. On the basis of the correlation of the 
various batteries with success we should expect to make the fewest 
mistakes in selection by using IQ + Q +Ts, but this is not absolutely 
certain. The correlations may not be uniform throughout the 
range: a battery may be highly selective at the top of the scale and 
less selective at the border-line. Further, the evaluation of a 
battery by correlation ignores the possibility of improving its 
selectiveness by considering, in border-line cases, the way in which 
the total score of a pupil is made up. 

Another serious limitation of the correlation technique is that, 
by itself, it gives us no information on the very important practical 
problem of fixing the pass-mark, a point which we shall postpone 
to a later chapter. 

For these and other reasons given later we decided to work over 
our data again, using an entirely different technique which throws 
light on the blind spots of the selection problem left by the correla¬ 
tion procedure. 

PROFILES 

For this alternative method it was necessary to have the detailed 
Qualifying and follow-up data for each pupil in a form in which 
any required fact could be seen at a glance. We therefore deci<^d 
to make graphical profiles of the type shown in fig. 8 (p. 93 ) 
each of the senior secondary pupils. As the technique is one which 

86 
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we recommend to future investigators we shall deal with their 
construction in some detail. 

At the left-hand side we have the pupil’s scores in the seven 
Qualifying tests and examinations. The next point, marked Av, 
gives the unweighted average of the seven. Then follow three 
points giving the primary teacher’s estimates of the pupil’s practical 
ability (Tp), industry (Ind.), and the Medical Officer s grading 
for health (H). 

On the right-hand side, the point between the first pair of 
vertical lines is the primary teacher’s forecast of the pupil’s success 
in the post-primary course actually selected. The next three points, 
marked I, 11 and III, give the pupil’s performance in the first 
three years of the post-primary course; and the last point indicates 
the head master’s final estimate of success. 

Necessity for using Standard Scores 

Ordinary marks or actual scores would be unsuitable for the 
construction of such profiles. Even if all scores were in percentages 
they would not be comparable and would be misleading. If, for 
example, a pupil had 70 per cent, in English and 70 per cent, in 
Arithmetic, the graph would give the impression that he was 
equally good in both; but this would clearly not be the case if 
the mean score in English were 70 per cent, and that in Arithmetic 
. 60 per cent. If, therefore, we wish the scores to be comparable, 
we must relate them to their respective means. Such a procedure 
will enable us to use a fixed line of means running right across the 
graph, so that we can tell immediately whether the pupil is above 
or below average in any particular measure. Even this method 
would be misleading if there were differences in the standard 
deviations. If a pupil is 10 per cent, above the average in 
English and in Arithmetic it does not follow that he is equally 
good in both; this depends on the standard deviations of the 
marks in the two subjects. 

For true comparability, therefore, we must take account both 
of the means and of the standard deviations, and this is done by 
the use of what are called standard scores. These are obtained 
by finding the pupil’s deviation from the average score of his group 
and dividing it by the standard deviation of the group. If the 
mean and standard deviation of the group are 60 per cent, and 
10 per cent, respectively, and if a pupil’s score is 70 per cent., his 
standard score is thus +ia. 
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The use of standard scores results in profiles in which all 
scores and all differences are comparable, and there is the further 
advantage that every point has a definite significance in relation 
to a normal distribution. If a pupil is placed at +i<r, this means 
that, in a normally distributed group, only about i6 per cent, of 
the pupils would have a higher score; if at — 1 < 7 , only about 
16 per cent, of the pupils would have a lower score. If his 
placing is +2<7, only about 2*3 per cent, of the pupils would 
have a higher score; and a standard score of +3a may be taken 
to indicate that the pupil is the best in his group. 

This, then, is the method which we adopted in making the pro¬ 
files. The horizontal at the middle of the graph is the average line, 
and the dotted horizontals are drawm at intervals of lo-. In fig. 8 
we see at once that the pupil is of outstanding intelligence. He is, 
relatively, not so good in English; and we have a confidence in 
the reality and size of the difference that we could not have had 
if we had used ordinary marks. 


Calculation of Standard Scores for Success Marks 

There are many technical difficulties in the determination of 
standard scores for our present purpose, which can best be 
illustrated by considering the marks. We wish to deal with 
the group as a whole, and the scores must therefore be strictly 
comparable for all pupils. 

The pupils are in five different senior secondary schools whose 
follow-up marks are not comparable. If we simply calculate a 
pupil’s standard score from the mean and standard deviation of his 
oion school, there is an error through assuming that the average 
standards of intelligence and attainment in all schools are the same. 
The standard gFiy scores have therefore to be corrected for 
differences in the average levels of the schools by arranging ^at 
they are given relatively to the mean and standard deviation of the 
complete group, that is, the pupils in all the schools if they were all 
marked on the same standard. All the gpAv marks are then 
comparable among themselves. 

We also wish the gFAv marks to be comparable with those for 
iFav and sFav; but the third-year group is more WgUy selected 
than the groups for the first two years on account of the dropping 
out of weaker pupils. If, therefore, a pupil has a standard score 
of + la in the third year he is better than one who has a score ol 
+ ICT in the first year. This difficulty can be overcome by making 
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the scores for all years relative to the first-year group. That is 
to say, when we give a transformed score of + Ict for a third-year 
pupil this means that, if the whole first-year group had remained 
to the third year, he would have been + la above the average of 
this group. Otherwise a pupil might appear to be falling back in 
his secondary work, whereas he is actually improving. 

Starting, then, from the raw marks we have to make three 
transformations: (1) into standard scores, (2) to ensure compar¬ 
ability between school and school, and (3) to ensure comparability 
between year and year. 

The foundation of the second and third corrections is the 
scaling of the follow-up marks of the different schools on the 
comparable Qualifying examination marks. That is to say, the 
transformed follow-up marks for a school will have the same mean 
and standard deviation as the Qualifying examination marks for 
the pupils of that school. We can now treat the follow-up 
marks for all the schools as comparable, and use the mean and 
standard deviation of the whole group for fixing the standard 
scores. It is, however, not necessary to scale the marks as a 
separate step. Each of the three transformations is linear; there¬ 
fore we can make all of them at once by means of a single equation 
which, though more complicated, is still linear.^ 


* Starling from the fundamental scaling equation (p. 44), and knowing the 
nature of the transformations to be made, there is no difficulty in arriving at the 
following formula for the complete transformation of a raw third-year score 
aF, for a pupil in School A into a comparable standard score aX,: 


Mq, - Mq, flj A^v, aMf, Aaq, 

— - + ■ - — ♦ A F% 

<^Qi <^Q, Affp, ffQ, Afp, ® 

where MQ,=mean Qualifying examination score of the whole ,Fat group 

for the five schools, 

M<3, = mcan Qualifying examination score of the whole ,Fav ffrouo 
for the five schools, 

= deviation of mean Qualifying examination mark of ,Fav group 
in School A from the mean Qualifying examination mark 
for the jFat group in all five schools, 

A<?p, = standard deviation of jFat marks for School A. 


The equation is linear, and when we put in the values of the 
reduces to 

aX,= -3-631 + .068 aF, 


Constants it 


V'TUa. “ comparable score 

‘"A™, are required for the first and second 
years, 15 equations in all are required* 
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Calculation of Standard Scores for Qualifying 

Tests and Examinations 


The first seven data on the left-hand side of the profiles are based 
on uniform tests or examinations, and are comparable within the 
December group and within the June group. We knew, however, 
that the level of intelligence and attainment was higher for the 
June than for the December group, a difficulty which we sur¬ 


mounted by the method of scaling, using P = 
bridge.^ 


IQ + EQ+AQ 
3 


as a 


Placing the Five-Point Gradings on the Profiles 

It was not necessary to place the gradings for industry, health, 
etc., with great accuracy, but it would have been misleading if we 
had placed, say, all the A gradings at the same a distance above 
the line of means. If, for example, l per cent, of the pupils were 
graded A in health and lo per cent, in industry, an A in health 
would indicate a higher standard than an A in industry, and the 
A point for health should be higher above the line of means. By 
a statistical procedure we can, from the distribution of the gradings, 
find the means and the ends of the ranges for the various categories, 
and these are indicated in the profiles by points with vertical lines 
through them.2 

' The following typical equation is that for converting Qe marks for the 
December group into comparable standard scores: 

a oMoe nOi ^ 

X = ——-H- 

<7p Op dctqa up DOQe 

where Qe = raw mark, 

X = standard score, 

DMQe = niean Qe mark for the December group, 
a =oMp - (D+j)Mp. 

* In these calculations we assume that the measures are normally distributed. 
We 5 nd the feet of the ordinates which would divide a normal distribution in 
such a way that the areas of the sections would correspond to the percentages 
of pupils graded in the various categories. Finally we calculate the means ot 
the sections of the curves, which will, of course, not usually be at the mi 
points of the ranges. For the primary teacher’s forecast and the final succ^ 
grading, where the letters had a definite objective meaning, we did not use the 
actual percentages in the various categories; we simply divided the base- ne 
of a normal distribution into five equal parts. 

References: K. J. Holzinger, Statistical Methods for Students in Education, 

S. ^’bawson. An Introduction to the Computation of Statistics. 

London: University of London Press, Ltd., 1933. F. 93 - 
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Calculation of the Average of the Seven Qualifying 

Tests and Examinations 

The average of the seven Qualifying tests and examinations is 
found by first adding the corresponding scores and then dividing 
by 7. Since each element in the numerator is in standard scores 
it might be thought that the averages obtained in this way would 
also be in standard scores; but this is not so. Their mean would 
certainly be zero, but the standard deviation would be less than 
unity by an amount which depends on the size of the inter-correla¬ 
tions. We had, therefore, to transform the averages into true 
standard scores by dividing by their standard deviation. 

standard scores for qualifying data not comparable 

WITH those for success 

There is one sense in which the data on the profiles are not 
comparable. A pupil may be above+ ia in the Qualifying data, 
but well below the average line in the success data. At first sight 
it would appear that he has fallen off very badly. It must be 
remembered, however, that his scores on the left-hand side show 
• zho\tx\it of the whole Qualifying group) while on 

the right-hand side his scores are below the average of the highly 
selected group that enters the senior secondary courses. 

To avoid confusion we shall now denote the standard deviation 
of Qualifying scores for the complete Qualifying group by sigma, 
using a for the standard deviation of the post-primary group with 
which we are dealing. 

The Short Horizontal Lines on the Profiles 

There are two short horizontal lines on the profiles. The one 
at the left-hand side is drawn horizontally at -h-y sigma, and may 
be regarded as an entrance pass-mark for the senior secondary 
courses. Pupils whose average scores lie above it have more than 
a 50 per cent, chance of success. Its determination will be 
explained later. If a pupil’s success scores lie above the short 
horizontal at the right, it may be taken that he has made good in 
the secondary course. 

Sample Profiles 

As these profiles have more human interest than the Tables of 
correlations with which we have hitherto wearied the non-mathe- 
matical reader, we shall now give a few samples which will afford 
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practice in their interpretation. We shall deal first with standard 
cases, where the pupil’s performance in the secondary course is in 
conformity with his Qualifying data. 


THE PUPIL WITH THE HIGHEST QUALIFYING SCORE 

Fig. 8 is the profile of a pupil who has fully borne out his 
early promise; he set us an easy problem at the Qualifying stage, 
and our answer would have been correct. This is a boy on whom 
fortune has smiled. He comes from a well-to-do home, has an 
imposing list of good qualities, ‘perseverance, originality, initiative 
and self-reliance, ’ and his primary teacher rightly described him 
as a boy of ‘great possibilities.* The fact that Music, Art and 
Bible are the only subjects of the secondary course which he 
regards as difficult is, we suspect, an indication of his tastes rather 
than of his abilities. He obtained his Leaving Certificate with 
great credit in 1941. 


THE PUPIL WITH THE LOWEST QUALIFYING SCORE 

For comparison we give, in fig. 9, the profile of the pupil who 
had the lowest average score in the whole Qualifying group. His 
follow-up scores are relative to the average of the backward class 
to which he was sent. The two form a striking contrast and should 
be of interest to any who still believe that all children are born 
equal in capacity, or that they should all have the same post¬ 
primary education. 


A CLEAR ADMIT 

The profile in fig. 10 is that of a pupil whom we should admit 
without hesitation. His standard follow-up scores of +’4o- to 
+ •50 are just what one would expect from his Qualifying tests. 


A BORDER-LINE ADMIT 

The next profile (fig. ii) is that of a girl of good ability in 
English but weak in Arithmetic. She is just above the border- 
line, and our hesitation as to her fitness for admission is increased 
by the low health grading. We find her, as we should expect, 
just above the success line in the secondary course. 
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Fig. 8.—The pupil with the highest Qualifyins ^core» 



Fxo. 9.—The pupil with the loweit Qualifyiog score* 
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Fig. 10.—clear admit. 


Qg E Tes Qp A Tas Av Tp Ind. H Fc. I 31 HI S 


Fig. II.—A border-line admit 
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A CLEAR REJECT 

The next (fig. 12) is a case where we should reject without 
hesitation. This girl is clearly below the border-line in intelli¬ 
gence, English and Arithmetic, and her primary teacher perhaps 
erred on the side of generosity in giving her a forecast of C. In 
the first two years of the secondary course she clung precariously 
to her hopeless position of about - la, plunged downward in the 
third, and left school to enter a business college. 



Fic. u.—A clear reject. 


A HOPELESS REJECT 

If the parents of this boy had seen his profile (fig. 13) one 

wonders whether they would have persisted in sending him to a 

senior secondary school against his own desire. Qualifying at 

13 years 9 months, he is weak in everything, has an E grading in 

mdustry, and only one marked personal quaUty, namely, laziness. 

Fiis later performance amply justifies the primary teacher’s 

pessimistic forecast. The secondary report states that the boy 

was fit only for a backward class; he had no interest in school- 

work, and finally started truancy in order to induce his parents 
to withdraw him. ^ 
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These graphs tell six human stories of children very differently 
endowed. They range from one pupil who, we trust, will be a 
leader in some department of culture, industry or commerce, to a 
number who seem destined to be merely hewers of wood and 
drawers of water. And while they all have their individual 
educational claims, these were certainly not met by a system under 
which all but one were allowed to enter a senior secondary school. 
The admission of pupils with no chance of success is harmful to 
the pupils themselves, and imposes a heavy burden upon the 
senior secondary teachers; but there is a third aspect which is 
sometimes overlooked. In these democratic days we think so 
much of the claims of the average and dull that we run the risk of 
forgetting those of the intellectual Hiu who will be the future 
torchbearers in cultural, social and industrial advance; and it is 
partly in the interests of this Hite that improvement in our methods 
of selection and guidance is necessary. 

DISCREPANCIES 

It must not be thought that the system is infallible. Later we 
shall reduce the discrepancies to proper statistical form, but mean- 
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time we present two cases where the reader, having had his lesson 
in forecasting, can make his predictions and find himself wrong. 

The two pupils in fig. 14 are at very nearly the same standard in 
the Qualifying data, and both qualified at 12 years of age. If the 
reader has profited by his lesson, he will have no hesitation in 
admitting both pupils, confident that they could be clear, but 
undistinguished, successes in a senior secondary course; if he has 



given some care to his preparaton he will predict, not only that 

they would be successes, but that in their success marks they should 
both have a score of about + •3<7. ^ 

The right-hand side shows how far he is in error; and his satisfac- 

fact th^t one of the pupils is a success is tempered by the 

Uct that this pupil IS far more successful than she ought to be 
Her success score, instead of being + .30, is well over + i .50. In 

ule^effrA I telf-reliant and specially 

lannual V l, course she took two foreign 

of L ® and German, and the head teacher’s e.xplanatiL 

fp thelm'f ‘he mother had" o b ng 

up the famUy without help. The pupil was e.xtremely eager to 

4 
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take full advantage of educational opportunities which, she realised, 
had been obtained for her with difficulty and sacrifice. 

The boy is an entirely different type, yet there was nothing 
in all our Qualifying records that could have saved us from the 
mistake of admitting him. He was a clear failure from the start 
of the secondary course, is reported to have been lazy and in¬ 
attentive throughout, and left after two years to become a salesman. 


TWINS 

The two following examples (fig. 15) do more credit to our 
system. They afford a picture of twin boys who show striking 
resemblances. Both have exactly the same IQ, the same marked 



special talents and the same personal qualities. Prediction was 
easy here, and proved correct. They are, however, both doing 
better in the secondary course than would have been expected, 
even with their high Qualifying average of over + 2 sigma. Both 
obtained their Leaving Certificates with great credit m 1941. 






PROFILES 


99 


HEALT}! 

Fig. i6 presents the dau for a case of no little difficulty, and 
is included to emphasise the point that factors other than examina¬ 
tion and test results should be considered in selecting and guiding 
pupils. It is that of a girl of a fine type, with everything in her 
favour except health. She is a perfectly clear admit, highly 
intelligent, interested in cultural pursuits, persevering and in¬ 
dustrious; a girl of originality and initiative, with special musical 



gifts and from a good home. But the Medical Officer’s health 
grading is D, and Ws confidential report shows that she has heart 
disease and anaemia. The report from the secondary school 
mentions that she is Melicate and often absent.’ Despite her 
physical handicap she has done weU in her secondary course 
though with distinct signs of falling off. 

It is doubtful whether a girl of this type should be encouraged 

to face the strain of a full Leaving Certificate course, with the 

temptation to venture on the still more intense strain of an honours 
degree course at a university. 
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STEADINESS AND VARIABILITY 

Fig. 17 raises a point which we have not yet had time to 
explore fully. Some pupils are remarkably steady in their perform¬ 
ance, others are extremely variable; and it is not always the steady 



^ IQ^ Qg E Tes Qg A Tos Av. Tp Ind. H Fc. I E HI S 

Flo. 17.—Steadiness and variability. 

pupil who is the good one and the variable pupil who is the poor 
one. The variable pupil, however, presents us with the more 
difficult problem, particularly if he is near the border-line. 

SPECIALISED ABILITY 

As a rule the pupil who is good in EngHsh is good in Arithm^ic, 
but occasionally we come upon instances like the fbUowing. 1 fte 
profile (fig. 18) is that of a girl with aspirations to become a music 
teacher, who has high Hterary gifts, but is very weakArithmetic 
This is no doubt partly responsible for her failure in the secon ^ 
course, where she found special difficulty with MaAemat.cs. As 
we shall see later, we should have rejected this pupd for her wea 
Arithmetic even though she had been considerably above Ae 

admission Une on the average. This, of course, 

that A the existing secondary courses she would be unlikely to be 
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Fig. t8.—A pupLl with specialiied ability. 



Ftc. 19. A pupil who has made a flop Id oqc subject* 
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a success. Whether, in a sound educational system, an aspiring 
musician, with literary gifts of the order of +i‘4 sigma, should 
be debarred from the chance of attaining her desire because of 
weakness in Mathematics, is open to question. 

A FLOP 

Our last illustration (fig. 19) at this point is that of a boy who 
has made a really thorough flop in his Arithmetic examination. 
With him no difficulty would arise, for he is a sure admit; but a 
collapse of this , kind in a border-line case might make all the- 
difference between admission and rejection if we had no means 
of convincing ourselves that the low mark was due to accident. 
He is not doing quite so well in his secondary course as we should 
expect, due, according to his head master, to laziness. 



CHAPTER XIII 


USE OF PROFILES TO DETERMINE THE NUMBER 
OF MISFITS BY DIFFERENT METHODS OF 

SELECTION 

General Procedure 

We know which pupils have turned out to be successful in the 
secondary course, and which have proved to be failures. If we 
take the standard average scores on the profiles as the basis of 
selection, and use a pass-mark of, say, + i sigma, we can now 
quickly deal the profile cards into two bundles, those whom we 
should admit and those whom we should reject. These two 
bundles represent the answers which, on this selection system, we 
should give to the 452 problems set to us by the applicants for 
admission to the senior secondary schools. But we now know the 
correct answers, and we can count our mistakes. 

These will fall into two categories: those whom we should have 
admitted but who have failed (admit-fails), and those whom we 
should have rejected but who have made good (reject-successes). 
These two categories together are the misfits. 

With this technique the possibilities of rapid experiment are 
almost limitless. Having ascertained the number of misfits with 
a pass-mark of +1 sigma, we can proceed to make a similar count 
for the same tests but with a pass-mark of +-5 sigma. We can 
also deal out the cards on the basis of, say, IQ alone, and find 

how the number of misfits compares with that for the complete 
battery. 

To the non-mathematical reader this very direct attack on the 
problem will doubtless carry greater conviction than the correla¬ 
tion results. It will also assist him in his interpretation of correla¬ 
tion coefficients. The statement that rgp =770 and tjq p =-691 
may convey little more than a vague idea that the examination 
IS a better basis of prediction than the inteUigence test, leaving 
him in doubt on two important points: 

*03 
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(1) Whether the difference is an important or a trivial one. 

(2) Whether a correlation of *770 means that the prediction will 

be virtually perfect, indifferent or good. 

He will find it more intelligible if we can tell him the numbers 
of mistakes which would result from the use of Qualifying 
examination marks and of intelligence quotient. 


SENIOR SECONDARY SCHOOLS 


Final Assessment of Success and Failure in the 

Secondary Course 


The use of the above method clearly demands a careful decision 
on the question whether the pupil has or has not made good in 
the secondary course; and this was reached in the following way. 
At the end of the third year of the follow-up each case was 
scrutinised separately, and we set aside those in which there was 
no possible doubt. The remainder, including those who had a 
success grading of C on the follow-up cards, were discussed in¬ 
dividually with the head teachers. If, in the opinion of the head 
teacher, the pupil had more than a 50 per cent, chance of obtaining 
a minimum Leaving Certificate in a suitable course in six years, 
he was counted a success; if less than a 50 per cent, chance, he was 
counted a failure. If a pupil was unlikely to make good through 
such causes as ill-health or bad home conditions, he was counted a 
failure even though the head teacher felt that he had the ability 
to make good under better conditions. 

The standard is not high, but we felt that, if a pupil reached it, 
no Education Committee could honourably exclude him from a 
senior secondary school. The qualification *in a suitable course 
was intended to meet a not uncommon case where the head 
master was of the opinion that the pupil was in the wrong course, 
but that he would have made good if he, or his parents, had made 


a better choice. , . 

The plan worked extremely well. The cards were arranged in 

order of merit of the follow-up marks, and at the top and bottom 
of the list the head teacher usually gave an immediate decision. 
In between came a doubtful band, much narrower than we had 
anticipated, where each case was carefully balanced, ^i^h 
siderable reference to mark books and reports. the 

head teacher deferred judgment until he had discussed the problem 
with various members of his staff. For such pupUs he was usually 
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good enough to send a full written report giving detailed reasons 
for his decision. 

One point that emerged clearly from these discussions was that, 
whatever variation there may be in the standards of marking of 
the individual schools, there is almost perfect agreement among 
the senior secondary head teachers as to the border-line which 
separates those who will gain the Leaving Certificate from those 
who will not. With the comparable follow-up scores before us 
we found that, after the interview with the first head teacher, 
we could tell almost exactly when the others would begin to 
hesitate, when they would sail into the smooth water of the sure 
fails, and where they would finally draw the line. We owe a deep 
debt of gratitude to the secondary head teachers for the courtesy 
and patient forbearance with which they gave us their help in this 
difficult task. 

Standard Follow-up Score for the Border-line 
BETWEEN Success and Failure 

When all the decisions had been made wc determined, by a 
method which will be explained later, the point on the scale of 
standard follow-up scores separating the successes from the 
failures. It was at — •5i<7 for the whole group and at almost 
exactly the same point for each separate school. 

Application of the Method of Misfits to test the Value 
OF the Qualifying Examination in Selection for 
Admission to Senior Secondary Schools 

Everything is now in readiness for the application of our post¬ 
mortem procedure; and wc proceed to show how many mistakes 
we should have made if we had used the Qualifying e.xamination 
as our test for admission. 

Fig. 20 is a picture whose story should be of considerable interest 
to all who have the responsibility for selection and guidance at the 
Qualifying stage, and the non-mathematician will find it more 
illuminating than the bare statement that rgp =-770. 

There is no horizontal scale; so the diagram, while showing 
whether the pupil was good or bad on the entrance test, does not 
show the degree of success which he attained in the secondary 
course—merely whether he was, or was not, a failure. The 
vertical scale is that of the standard Qualifying examination scores. 
The pass-mark of the examination is taken as +*6o sigma. 

4 * 



STANDARD SCORE IN QUALIFYING EXAMINATION 
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Fig. zo.— 


ADMIT-SUCCESS 


ADMIT-FAIL 





•Miltakes which would have been made if the Qualifying examination bad 
bccQ uttd zi tntTZDCc tcii for lonior tocondiiy couric*. 


We now count our misfits with the following results: 


Admit . 
Reject . 


Total 


Successes 

283 

21 

304 


Failures 

38 

74 


112 


Total 

321 

95 

416 ^ 


1 In order to ensure comparability of the rcsulu which foUow it was necessary 
to omit pupils who had absences from any of the seren tests. 
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Out of the 416 applicants we should have admitted 321, and of 
these, 38 would not have made good. We should have rejected 
95 pupils, 21 of whom would have been wrongly debarred from 
the opportunity of taking a senior secondary education. The 
number of mistakes we should have made is 59 out of the total 

of 416. 


Range of Entrance Attainments of the Misfits 

One of the most interesting points brought to light by the 
diagram is the scatter of the misfits on the entrance attainment 
scale. Pupils with a Qualifying score as high as +i’50 may be 
failures; pupils as low as --lo may make good. The band of 
uncertain prediction lies between the lines CD and EF. 

Some have aspired to a selection system where no pupils would 
be admitted to a senior secondary course who would not make 
good, and are disposed to be critical of the presence of any tailures. 
The diagram shows that it would be possible to satisty their 
demands. We should fix the pass-mark at CD; but the purity 
of the secondary classes would be purchased at the cost of denying 
a secondary education to no less than 223 pupils fit to profit 
by it. Others, taking a more humane view, desire to reject no 
pupils who would ultimately make good. Their demands could be 
satisfied by lowering the pass-mark to EF, but at the cost of 
admitting 95 pupils who would clog the educational machine and 
suffer the depressing effects of early failure. 


The Pass-Mark 

This leads to the important problem of the correct fixing of 
the pass-mark, and we use the diagram to introduce the reader 
to the method by which we studied it. With a pass-mark AB 
we have 59 misfits. Suppose we raise it to + *70, we reduce 
the admit-fails by 4, but we increase the reject-successes by 8, 
giving a net increase of 4 misfits. On tabulating the results of 
raising or lowering the pass-mark in this way we come upon an 
important law. 

The law, which we shall find to be general, is that for any 
entrance test there is a pass-mark at which the number of misfits 
is a minimum—clearly a law of fundamental importance. 

One other notable fact that emerges from a study of Table 
XXXVII is that, at the point where the misfits are a minimum, 



io8 


SELECTION FOR SECONDARY EDUCATION 


TABLE XXXVII 


Effect of changing the pass-mark on the actual number of misfits 

(Entrance test, Q) 


Pass-mark 

Admit-fail 

Reject-success 

Misfits 

+1 -60 sigma 

0 

223 

223 

+ I-40 

4 

186 

190 

+ 1-20 

9 

130 

139 

+ 1-00 

18 

81 

99 

+ -So 

27 

43 

70 

+ 70 

34 

29 

63 

+ 60 

38 

21 

59 

+ -50 

SI 

14 

65 

+ -40 

60 

10 

70 

+ -20 

69 

6 

75 

•00 

86 

3 

89 

- *20 

95 

0 

95 


the number of admit-fails is considerably greater than the number 
of reject-successes. 


The Correct Pass-Mark 

It will now be convenient to consider the general question of 
the position of the pass-line. This is a problem on which science 
can throw some light, but the complete solution transcends 
scientific facts. Our finding depends to some extent on whether 
we think it does more harm to reject a success than to admit a 
failure. If the philosopher will solve that conundrum, and put 
his answer in mathematical terms (for example, that the rejection 
of a success is twice as harmful as the admission of a failure), the 
scientist will do the rest. 

As we know of no authoritative pronouncement on the pomty 
we shall assume that the two types of mistake are equally harmful. 
That being so, we suggest that the pass-mark is drawn correctly 

when the number of misfits is at the minimum. ^ 

We shall find later that the minimum always comes at a dehmte 

point, at which: 
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The pupil who is admitted has more than a 50 per cent, chance 
of succeeding; the pupil who is rejected has less than a 
50 per cent, chance of succeeding. 

In all cases likely to arise in Scotland, the number of admit- 
fails will be greater than the number of reject-successes. 

If we err, therefore, it is on the side of giving the child his 
educational chance. 

Another possible pass-mark is the point at which the number of 
admit-fails is equal to the number of reject-successes. This has 
a specious air of fairness, but a glance at Table XXXVII will show 
that it would give a larger number of misfits, and its unsoundness 
will be more clearly disclosed later. 

We shall in future always refer to the point at which the number 
of misfits is a minimum and the child has just an even chance of 
success as the cornet pass-mark. 

We now proceed to compare the predictive value of the various 
tests, examinations and batteries by finding the number of misfits 
each would give if it were used with its own correct pass-mark. 


Method of Tabulation 


Tables of the following form were prepared for all groups, 
senior and junior secondary, backward and retained. 


Class 

interval 

Senior 

Leaving 

Certificate 

Junior 

Leaving 

Certificate 

Day 

School 

Certificate 

(Lower) 

Shorter 

post- 

primary 

course 

Backward 

class 


S* F-f 

S F 

S F 

S F 

4 • 

4 « 

S F 

+ -40—49 

4 »7 

7 *4 

21 0 

4 « 

4 # 

21 0 

0 0 


(sigma) 11 11 II II II 

• S=Success t F = Failure 

In the case of the Junior Leaving Certificate a pupil was graded 
a success if, in the opinion of the head teacher, he had more than 
a 50 per cent, chance of obtaining the Certificate in three years. 
A similar test was applied for the Day School Certificate (Lower), 
the time allowed being two years. 
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Determination of the Minimum Number of Misfits 

FOR A Test or Battery 

There are two difficulties in determining the minimum number 
of misfits. In the first place, the minimum may lie within a class 
interval; in the second place, the numbers within class intervals 
of •! sigma are small, and there is consequently a degree of un¬ 
steadiness due to the chance fluctuations of small numbers. We 
must therefore try to introduce some of the steadying effect of 
the numbers above and below the pass-line. 

If we take Table XXXVII as our illustration, the actual number 
of misfits is 59 with a pass-mark of +-6o sigma. To get a fairer 
estimate of the general tendency of the Table we could plot the 
number of misfits against the pass-marks, smooth the curve, and 
find where the smooth curve marks the minimum. 

Generally speaking, there are two ways of smoothing curves: 
mathematical methods, like the method of least squares, and 
graphical methods. The first method is the soundest but involves 
very heavy calculations. The second method can be carried out 
rapidly without calculation, and works very satisfactorily with 
certain types of curve. The essence of such methods is that we 
replace each point on the curve by the weighted average of it and 
of the points above and below.^ 

The disadvantage of the graphical method is that it tends to 
make a concave curve a little less concave, and all the curves of 
misfits are of this type. To avoid this we tried various expedients, 
but finally decided to plot separately the curves for admit-fails 
and reject-successes. These, as will be seen from fig. 2 i, Were 

much more suitable for graphical smoothing. 

The figure shows how the smoothing removes the unsteadiness 
which comes from small numbers. The number of misfits for 
any pass-mark is found by reading from the graphs the numbers 
of admit-fails and reject-successes, and adding. 


Determination of the Correct Pass-Mark 

The correct pass-mark was determined in mo ways. At the 
correct pass-mark the number of misfits is a minimum; therefore 

1 If we have a scries of points y.. y., y„ . . . y. to find the 
corresponding to y„ a ruler is placed through y, and y,. ^he nud-pomt of th 
vertical distance benveen y, and the ruler gives the smoothed point. In this 

way we replace y, by In refractory cases the process can be 

repeated vvith the smoothed points. 



PERCfNTAGE OF FAILURES 



Fic. 21.—Smoothing of curve* of admit-fails and reject-successes 

(Battery IQ + S + Ts). 
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we can find it from the above-mentioned graphs. The pupil has 
an even chance of success; and the correct pass-mark will be the 
point at which the 50 per cent, line cuts the curve formed by 
plotting the percentage of failures for each class interval. This 
is shown in fig. 22. 

Figs. 21 and 22 are both in respect of the battery IQ + S+Ts, 

and by both methods the correct pass-mark comes out to be 
+ •673 sigma.1 


Correct Pass-Marks for Various Tests and Batteries 

The pass-marks obtained in these two ways for all tests and 
batteries are given in Table XXXVIII. The batteries are arranged 
in order of the correlation coefficients, which are given in the first 
column. 

TABLE XXXVIII 


Correct pass-marks in terms of sigma for all tests and batteries 


Battery 

Corre¬ 
lation 
with F 

Pass-* 
mark 
{50 per 
cent* 
chance) 

Pa5$- 

xnark 

(mini. 

mum 

misfiti) 

d 

Average 

pa$ 9 * 

mark 

Average 

pa»s- 

mark 

smoothed 

Theoretical 

pas5-*mark 


CO 

(0 

( 3 ) 

( 4 ) 

(5) 

(6) 

( 7 ) 

IQ+Q + Ts . 

•S04 

•696 

.707 

-•on 

•701 

.70 

•68 

IQ + Q + S -f- Ts 

•800 

•700 

• • 


•700 

.70 

•67 

Q + S + Ts . 

.790 

•701 

•701 

•000 

•701 

.69 

•66 

IQ + Q . 

.786 

•676 

•644 

+ •032 

•660 

•69 

•66 

Q +T8. 

•783 

<680 

■713 

-•033 

•696 

•68 

•66 

IQ + Q + S . 

•783 

.654 

•676 

-•022 

•665 

•68 

•66 

IQ+S+Ts . 

•782 

•673 

.673 

•000 

.673 

•68 

•66 

IQ + Ts. 

•779 

•680 

*646 

+ .034 

•663 

.67 

•65 

Q +S . 

•774 

<660 

•649 

+•011 

•654 

•66 

.65 

Q. 

.770 

•631 

•631 

•000 

•631 

•66 

.65 

S +T8. 

•764 

*666 

•666 

•000 

.666 

.65 

.64 

IQ+S . 

•736 

*572 

•647 

—07s 

•609 

•62 

•61 

Ts 

♦720 

•642 

.566 

+ •076 

•604 

•61 

•59 

S . 

•698 

•633 

•666 

-•033 

.649 

•58 

•56 

IQ 

•691 

•475 

•475 

•000 

•475 

•57 

•55 


The discrepancies between the two methods of finding the 
pass-mark, shown in column (4), are not great, and exceed '034 
sigma only for IQ + S and for Ts. To get a more reliable estimate 

1 In determining the correct pass-mark it has to be kept in mind that the 
Standard deviation of IQ + Q + Ts is less than unity- 
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of the pass-mark the two measures were averaged, giving the 
results in column (5). 

TilE HIGHER THE FOLLOW-UP CORRELATION FOR THE BATTERY 

THE HIGHER THE CORRECT PASS-MARK 

Column (5) reveals the important law that the higher the 
predictive value of the battery the higher should be its pass-mark. 
In the general discussion, to which this chapter is an empirical 



CORREtATlON 

Flo. 23.—Fall of pau-mark with fall of correlation: theoretical curve 

and smoothed actual cur\’e. 


preamble, the reason for this will be given, and we shall find that 
the theoretical curve of fall of pass-mark with fall of correlation 
is smooth. We may therefore assume that we should get more 
reliable estimates of the pass-marks by smoothing the actual curve. 
The result of the process is shown in fig. 23. 

No special significance attaches to the fact that the theoretical 
curve is a little lower than the actual one: this is due to the skew¬ 
ness of the distribution of the senior secondary group. It is 
required at the moment merely to act as a check on the general 
sweep of the actual curve; and when we consider the size of the 
vertical scale it is clear that the agreement is satisfactory. We 
therefore take the points obtained from this smoothed curve as 
the best estimates we can get of the pass-marks for the batteries- 
and these are given in column (6) of Table XXXVIII. The pass- 
marks obtained from the theoretical curve are given in column (7). 
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For assessing the significance of the small possible errors in the 
determination of the pass-marks we give the following guide: 

An error of -03 sigma in fixing the pass-mark would increase 
the misfits by about i per cent. 

An error of *02 sigma would increase the misfits by about 
I in 200. 

An error of ‘Oi sigma would increase the misfits by about 
I in 500. 

Number of Misfits for the Various Tests and Batteries 

We now come to the misfits for the various batteries, full details 
of which are given in Table XXXIX. For the sake of those who 
regard all statistical manipulation with suspicion we give, in 
column (2), the actual minimum misfits obtained from the dis¬ 
tributions with a class interval of *1 sigma. These, however, are 
not the exact minima, which would usually fall within a class 
interval. In columns (3), (4) and (5) we give the minima obtained 
as explained above from the smoothed curves. 


TABLE XXXIX 

Number of misfits for the various tests and batteries 




Corre¬ 

lation 

Actual 

miDi* 

Actual mioimum misfits 
(smoothed) 

Minimum misfits for 
normal correlation 

Corre¬ 

lation 

corre- 

tpond- 

Battery 


with 

F 

mum 

misfits 

Admit** 

fail 

Reject- 

success 

Misfiti 

Admit- 

fail 

Reject- 

success 

Misfits 

ing to 
actual 
mitfits 

(col. (5)) 



CO 

(0 

(3) 

(4) 

( 5 ) 

( 6 ) 

(7) 

( 8 ) 

( 9 ) 

IQ+Q+Ti . 
IQ+Q+S+T* 
Q +S+Ti . 

IQ+Q . 

Q +T« . • 

IQ+Q+S 
IQ+S+T$ . 
IQ+Tt . 

Q +S . 

Q. 

S +T» . 

IQ+S . 

T» 

S . 

IQ 

•804 

•800 

.790 

•786 

.783 

•783 

•78a 

•779 

•774 

770 

.764 

•736 

•720 

•69S 

•691 

56 

55 

56 

55 

59 

58 

58 

55 

57 

59 

56 

6z 

59 

64 

69 

■^9 

■Llfl 

K&ljB 

^^9 

i8*4 

19- 4 

20- 0 

19*0 

20*1 

21*9 

19*2 

17-0 

19*1 

20*4 

20*t 

25-1 

13*0 

27-4 

22*2 

% 

56-5 

56- 5 

57 - 4 
57-5 

59-0 

59.1 
587 

58.1 
59-3 

6i'S 

6o*3 

64'I 

6m 

68*0 

69-3 

44.0 

44*5 

46*0 

467 

46- 9 

46.9 
47.0 

47 - 5 

48- 1 

48.9 

49.6 
53-7 
55-8 
58-6 

59.6 

1 

24-4 

247 

24-8 

249 

24-9 

24-9 

24 - 9 
25*0 
25'0 

25- 1 
25-2 
25-5 
* 5-4 
25.4 
25.3 

68-4 

69*2 

70*8 

71*6 

7 I'8 

71*8 

719 

72*5 

73-1 
74.0 

74*8 

79-2 

8l>2 

84-0 

84*9 

•802 

•802 

■793 

792 

780 

•779 

782 

787 

•777 

760 

.770 

736 

763 

•701 

•690 














USE OF PROFILES TO DETERMINE MISFITS 1.5 

There is no serious discrepancy between the actual minima and 
those obtained from the smoothed curves except perhaps in th 
° Ts-i-Ts where a study of the curves leaves no doubt 

Tto the grea’ter soundness of the smoothed result^ The way 
in which fhese deviations arise may be explained by reference 
to fig. 21. There the minimum is indicated by a vertical me 

near^which the actual point on the admit-fail 
the smooth curve. For the reject-success curve the opposite 
holds. Here, therefore, the actual minimunt would be close t 
the smoothed minimum; but if the actu.al points had fallen 
below the smoothed curves in both cases the discrepancies would 
have been additive, and the difference considerable. 

The numbers of misfits range from 56-5 for the best b.attery to 
6q-t for the poorest. These should be considered in relation to 
M2, which is the number of misfits we should have h.id if we 
had dispensed with the entrance examination and admitted all the 
candidates. Our poorest battery gives a correlation with t ot 
.691. By methods described in Chapter XIV we can estimam the 
numbers of misfits for batteries giving lower correlations. These 

are as follows: 


Correlation with F 
•6 

5 
4 
3 
2 
1 
o 


Minimum number of misfits 

82 

94 

105 


111 

112 
112 
112 


From these it will be noted that entrance examinations giving a 
correlation with F of less than are of very little value. 


Orders of Predictive Value based on Correlations 

AND ON Misfits 

The relationship between the two orders will be shown later 
in graphical form. Meantime, an inspection of columns (i) and 
(5) of Table XXXIX will show that the conformity between the 
two is fairly close. The first six batteries are in the same order 
on both bases; and the differences which cause the later dis¬ 
crepancies in order are small. 

It is, of course, not necessary that the orders should be the 
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same. The correlations are based on the whole of the data, 
and give the order of discriminative capacity of the batteries 
throughout the scale: the misfits give the order of discrimination 
at the border-line. As far as practical application is concerned 
we might put the position in this way. The order of misfits 
is the more significant for the Education Committee which is 



Fig. 24.—Comparison of actual and theoretical number of misfits 

for difiterent sues of correlation. 


concerned merely with the selection of pupils for admission to 
senior secondary schools: the order of correlations is the more 
significant for the secondary head master who will use the results 
of the entrance tests for arranging his entrants into homogeneous 

groups. , . 

In Chapter XIV we shall see that it is possible to calculate 

theoretically the number of misfits for any size of normal correla¬ 
tion. For comparison, these theoretical values are given in 
columns (6), (7) and (8) of Table XXXIX, but the relationship 

is best shown by the graphs in fig. 24. 
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The fluctuations are somewhat less for the smoothed actual 
curve than for the actual curve; but they are far from completely 
eliminated, and give the impression that the deviations are real 
and not chance variations. The point A correspon.^ to Q, and 
B to Ts; and while Q has the higher correlation, Ts gives the 
fewer misfits. It may well be that this is no mere accident. 
Q may get its high correlation by properly placing the better 
pupils, while the teachers’ estimates may be particularly dis¬ 
criminating at the border-line. 


Comparison between Actual and Theoretical 

Numbers of Misfits 

The theoretical and the smoothed actual curves differ in two 
ways. In the first place, the actual numbers are all much less 
than the theoretical. This is no doubt due, in small part, to the 
under-estimation of the correlations through a cumulation of 
small losses, but mainly to the fact that the distribution of the 
senior secondary group is tailed. In the second place, the actual 
curve has a slightly smaller slope than the theoretical. This is 
related to the first difference, as can be seen if we imagine the 
curves completed to the ends, r = o and r = i. 


Correlations which would give the Actual 

Numbers of Misfits 

Most of our data are in the form of correlations, and ultimately 
we shall gather together all our evidence in order to get a final 
order of predictive value for the various tests and batteries. It 
would therefore be convenient if we could put the order based 
on misfits in this comparable form. Since the theoretical and 
the actual numbers of misfits are not the same, we cannot do so 
by finding the normal correlations that would give the number 
of misfits for each battery. The only method is to draw a smooth 
curve through the points for the actual smoothed curve and read 
the correlations from it. In so doing we are not assuming, 
contrary to what has been said above, that the actual smoothed 
curve should have the same smooth sweep as the theoretical one. 
We retain the order of the misfits and simply put the data into 
another form comparable with correlations. These fictitious 
correlations are given in column (9) of Table XXXIX. 
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Composition of Misfits 

From Table XXXIX it will be seen that there are considerable 
fluctuations in the proportions of admit-fails and reject-successes. 
This is due to the fact that while the total number of misfits is 
not very sensitive to small changes in the placing of the border¬ 
line, the proportions are. The numbers in the Table should 
therefore be taken as giving merely a rough indication of the 
proportions. 


JUNIOR SECONDARY SCHOOLS 

Distribution of the Junior Secondary Group 

The fact that the distribution of the senior secondary pupils 
is tailed caused certain differences between the theoretical and 
the actual results. As will be seen from fig. 57,^ the distribution 
of the junior secondary group is more nearly normal and we 
should anticipate that the correspondence would be closer. 


Qualifying Pass-Mark for Success in the 
Junior Leaving Certificate 

As for the senior secondary courses, we determined the point on 
the standard scores for IQ+Q + S+Ts at which the pupil has a 
50 per cent, chance of success in the Junior Leaving Certificate. 
For this battery the correct Qualifying pass-mark is as follows: 

Pass-mark Pass-mark Theoretical 

(50 per cent, chance) (minimum misfits) pass-mark 

+ *595 sigma + *595 sigma + -597 sigma 

The border-line for success in the Junior Certificate is only 
about ■! sigma lower than that for the Senior Certificate. The 
standard for the Certificate is therefore fairly high. 

Should anyone wish to find the correct pass-mark for one of the 
poorer batteries he will get a very close approximation by sub¬ 
tracting -I sigma from the appropriate number m column (6) ot 

Table XXXVIII. 


* p. 203. 
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Number and Composition of Misfits in Selection of 
Pupils for Junior Leaving Certificate 

Using the methods already described, we can find the number 
of mistakes we should make if we used the battery IQ + Q + S + Ts, 
with its correct pass-mark, to select pupils who should take a 
Junior Leaving Certificate course in junior secondary schools. 
The total number in our junior secondary group was 1,975. 

Admit-fiils Reject-successes Misfits 

Actual . . 164 25^ 422 

Theoretical. . 143 276 419 



CHAPTER XIV 

GENERAL PRINCIPLES OF SELECTION TECHNIQUE 

We have, by empirical procedure, come upon several laws govern¬ 
ing the operation of the selection process. Since these might 
be specific to our data, the time is ripe to gather the various 
points together in a theoretical treatment which will show how 
far they are general. 

The present chapter is an attempt to systematise a collection 
of principles which we encountered from time to time in working 
up the Inquiry data. Starting from peculiarities of our results 
we made various explorations into the field of theory, and 
found that a surprisingly large number of laws could be deduced 
mathematically without experiment. We felt that our own 
picture of the selection situation was clarified in this way, and we 
hope that this chapter may do the same for our readers. Certainly 
it brings out principles which should be in the minds of all 
responsible for selecting pupils for secondary education or 
awarding bursaries or free places. 

Our honesty in recommending the non-mathematical reader to 
omit Chapter VI may give him greater confidence in our advice that 
he should read this one. He should not be deterred by the fact 
that, at places, it has a mathematical appearance; this is deceptive. 
A certain amount of difficult mathematics undoubtedly is involved 
in some of the operations; but we believe that, by taking the 
results of these operations for granted, he will be able to foUow 
the drift of the reasoning and understand how the various 
principles are reached. 

The Scatter Diagram 

Fig. 20 ^ shows whether the pupil is a high or a low admit, 
but gives no indication whether he was a distinguished or merely 
a border-line success in the secondary school. We can display 
the results in a way that wiU give us both pieces of information 

by a figure of the following type^____ 

* P. io6. 
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Fic. 25.—Scatter diagram showing position of pupil both in entrance test 

and in success. 


The pass-mark on the entrance test will be represented by a 
horizontal line like AB, and the line separating successes from 
failures in the secondary course by a vertical line like CD. Pupils 
in the quadrant CEB will be admit-successes; those in the 
quadrant BED reject-successes, and so on. If a pupil’s position 
is in cell P we know that his entrance test score was between 
70 and 80 per cent., and that his success score was only between 
40 and 50 per cent. 

It is from scatter diagrams of this kind that correlations are 
calculated, and the entries usually fall within an ellipse. Along 
narrow ellipse means that a pupil with a high score on the entrance 
test generally has a high score in success, and vic^ versa. The 
correlation is therefore high. A broad ellipse, approaching 
circular shape, indicates a low correlation; a long narrow ellipse 
lying along the 90-90 diagonal would signify a high negative 
correlation. 

In any row or in any column the numbers are high in the 
middle and decrease towards the ends, 
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The Normal Bivariate Surface 

The title is forbidding, but the notion is simple. If we were 
to erect a square tower on each cell, the height of the tower 
being proportionate to the number of the pupils in the cell, we 
should get a solid which is high towards the centre and lower 
towards the sides and ends; and if we imagine the numbers to 
be increased and the class intervals diminished, we should in the 



Fio. 26.—Schematic representatioD of the bivariate surface correipondiog to the 

correlation between entrance test and success. 


end get a smooth surface resembling that of a policeman’s helmet. 
If the distributions in the rows and columns are normal, the 
result is called a normal bivariate surface, and it has the following 
properties. Each vertical section, whether parallel to the x- or 
the y-axis, is a normal distribution curve, and each horizontal 
section is an ellipse. 

In view of the latter property we can use an ellipse as a shorthand 
representation of the bivariate surface for any particular correla¬ 
tion. If the ellipse were at a distance of about 30- from the centre, 
it would include practically all the cases, and might be regarded 
as the base of the solid. Our standard figure for further discussion 
is thus of the type shown in fig. 26. 
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When the correlation has a bivariate surface with the above- 
mentioned properties we speak of it as a normal^rrelation 

AB is the pass-mark on the entrance test; CD the line dividing 
successes from failures in the secondary course; O is the centre 
of the ellipse and PQ the major axis. 


Basis of the Mathematical Treatment 

Our problems now become amenable to mathematical treat¬ 
ment, for tlic equation of the normal bivariate surface is known. 

If we refer to the scatter diagram in fig. 25 we see that the 
number of pupils in the quadrant CEB is the volume of the solid 
cut off by vertical planes through CE and EB. Similarly, the 
number of misfits is the sum of the volumes CEA and DEB. 
Since the equation of the surface is known these volumes can be 
determined mathematically.^ 


Problems of Selection and Guidance 
There are three main problems at the threshold of secondary 
education: 

Prediction of the success of an individual pupil in the 
secondary course. 

Selection of pupils for a definitely limited number of scholar¬ 
ships or free places. 

Selection of pupils who are fit for secondary education, the 
number not being definitely limited. 


PREDICTING the SUCCESS OF AN INDIVIDUAL PUPIL 

To make the problem more definite we take two sample figures. 
The average correlation between an ordinary entrance examina¬ 
tion and success in the secondary school in four recent careful 
investigations (three in England and one in Australia) is +-34. 


^ If OF expressed in terms of the standard deviation of Af = A, and OG expressed 
in terms of the standard deviation of y = ky then the proportion of misfits is 
pven by 


i ( f® 




e 2 ( 1 -r*) 


i:r 


-(**-2rzv + y*) 


2(1-r*) 


dxdy 


) 


References: G* Udny Yule and M. G. Kendall, An Introduction to the Theory 

of Statistics^ Chapter xii, 

K. Pearson, Tables for Statisticians and BiometricianSy Part IL 
First edition. London: Cambridge University Press. 
Mathematical Introduction, p. lii. 
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Our correlation for the best batter/, one of the highest ever pub¬ 
lished, IS + -So. With what assurance could we, in these two cases, 
predict a pupil s success if we were given his score at entrance? 

The methods of dealing with this problem are well known and 
are dealt with here mainly for the sake of comparison with what 
follows in regard to group selection.^ They are based upon the 
fact that if the correlation between two variates is known we can 



Fic. 27.—Significance of different sizes of correlation in predicting the success 

of an individual pupil. 


find the standard error of an estimate of the second when given 
the first.2 

Clearly, if the correlation were +1, the error of our estimate 
of a pupil’s success, given his entrance test score, would be zero. 
If the error of estimate by pure guesswork ® were 100, we can, 
by this method, give the error of estimate for any size of correla¬ 
tion in the form of a percentage. This is done graphically in 

can now find the answers to our two questions. If the 
correlation between entrance examination and success is +‘ 34 > 


fig. 27 
We 


• T. L. Kelley, Statistical Method, Chapter viii; Interpretation of Educa¬ 
tional Measurements. New York: World Book Company, 1927* Chapter vu. 
A. S. Otis, Statistical Method in Educational Measurement. New York: World 
Book Company, 192^. Chapter xvii. 

• ai a i 

• By ‘pure guesswork* we mean that we predict with no knowledge of the 
pupil’s entrance test score, or on the basis of an entrance test whose correlation 
with success is zero. In both cases we would predict the average of the group. 
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the error of prediction of the success of an individual pupil v«uW 

be about 94 per cent, of the error by pure 
with our or™ highest correlation of -8 it would be 60 per cent 
These results show the uncertainty of any guidance given 
pupil solely on the basis of examinations and tests, no matter how 

complete or carefully conducted. 


SELECTION FOR SCHOLARSHIPS AND FREE PLACES 

Here we are interested, not in the individual pupil, but in the 
number of wrong awards which we should make with different 

systems of selection. 

A Sample Case: Award of lOO Scholarships with 365 Candidates 

We use this as an illustration of the mathematical method; and 
we shall show how, if we know the correlation,' we can te ow 
many wrong awards we should make. 



success IN SECONDARY SCHOOL 
Fig. 28.—Award of 100 icholanhipj with 365 candidate!. 


The pass-mark AB is fixed for us because it must cut off exactly 
100 applicants. Since we are assuming normal correlation, we can 
easily find how far AB must be above the mean of the examination 

' At this and various other points in the Report we have to bear in mind the 
distinction between a correlation based upon a truncated distribution and the 
correlation based upon the complete distribution. 
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distribution if it is to cut off this numberA This distance, OG, 
in terms of the standard deviation of the distribution, is denoted by 
k In this case OG = k~ -dcr. The volume ACB then indicates 
the pupils who actually get scholarships. 

Since these scholarships are awarded for secondary education, 
we may assume that the intention is to award them to the loo 
pupils^ who would be most successful in the secondary course. 
The line dividing those who should have got scholarships from 
those who should not is therefore a vertical line CD which cuts off 
the top loo as far as success is concerned. This must be *6(7 
above the mean, so that OF = h= +- 6 a. AB and CD must meet 
on the major axis of the ellipse, and the volumes AEC and BED 
must be equal. 

We can now interpret our picture: 

The pupils in volume CEB got scholarships, and they come into 
the top loo in success. They therefore merited their awards. 

The pupils in volume AED were correctly refused scholarships. 

The number of pupils who got scholarships in error is the 
volume AEC. 

The number of pupils who were unjustly refused scholarships 
is the volume BED. 

The number of mistakes which we should have made is the sum 
of the volumes AEC and BED. Given the size of the correlation 
it is possible to calculate all the volumes, as already explained, with 
the following results: 

TABLE XL 

Award of lOO scholarships with 365 candidates 


Method of award 

Correct 

awards 

Wrong 

awards 

Wrong 

refusals 

Total 

mistakes 

Pulling names out of a hat * 

27 

73 

73 

145 

Examination : 
r= +‘34 

42 

58 

58 

116 

f = + *50 • • • 

50 

50 

50 

ICO 

f* = + *80 . ♦ • 

69 

31 

31 

62 


• The numbers in this row are the most probable numbers. 

* This is done by means of Tables of the Probability Integral. Cf. K. Pearson, 
Tables for Statisticians and Biometridans, Part I. Table II. 
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These are again disappointing results, particularly when one 
considers that tew scholarship examinations, judging from the 
published results, are likely to have a correlation with success as 
high as -5. They will bring comfort only to those who at some 
time have been unsuccessful in such competitions. 

Curve showing the Significance of Correlations of Different Sizes in 
the award of Scholarships 

In dealing with the general problem it would be interesting, and 
sometimes helpful in practice, to have a,curve similar to that in 
fig. 27 which would show the fall in the number of rnistakes 
with increase of correlation between the scholarship examination 
and success. 

In making such a curve there is this difficulty, that the number 
of mistakes depends not merely on the size of the correlation but 
also on the ratio of the number of scholarships to the number of 
candidates. We require therefore not one curve but many—one, 
indeed, for each ratio. 

Given the ratio it is easy to make the curve. The ends are fixed. 
If r = +1, the number of mistakes is zero; if r = o, the number 
is the same as we should get by chance selection. The inter¬ 
mediate values can be calculated by the method described above.^ 
The results for different ratios of scholarships to candidates are 
shown in fig. 29. 

The curves show that, the predictive value of a correlation 
coefficient rises most rapidly when the number of scholarships 
is half the number of candidates. In this case the number of 
mistakes is reduced by one-half when the correlation is about ‘7. 
If the number of scholarships is very small, say *5 per cent, of the 
number of candidates, it would take a correlation of over *9 to 
effect the same reduction. 

For practical purposes a nest of different curves is not very 
convenient, but we can make a simplification in the following way. 
By reference to the symmetry of fig. 28 we see that the number 
of mistakes would be the same when h = k— —‘60 as it is when 
h = k= +'6 <t. Further, the number of scholarships or bursaries 
awarded is not likely, at any rate at the Qualifying stage in Scot¬ 
land, to be less than 16 per cent., or more than 84 per cent., of the 
total number of candidates. All cases between these limits fall 
* When the number of scholarships Is half the total number of candidates 

there is a simpler method. In this case proportion of mistakes = "cos” V. 
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within the narrow band enclosed by the two lower curves. A 
curve drawn midway between them will therefore give a fairly 



_ 50 y« OF GROUP AWARDED SCHOLARSHIPS 
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«— •• 
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Fig. 29.—Significaoce of different size* of correlation n award of icholarsiupi: 

Exact curves for different percentages. 



Fig. 30.—Significance of different sizes of correlation in award of scholarships; 

Approximate ilogte curves 


close approximation to the number of mistakes for all the cases 

likely to arise in practice. ^ 

The curve determined in this way is shown in fig. 30. It is 
carefully drawn, and can be used in practice to find the number ot 
mistakes for any given size of correlation. 
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Example of the Use of the Curve in Jig. 30 

To illustrate the way in which the curve could be used we take 
the sample case given above, where we know the correct answer. 
Here we have to award 100 scholarships with 365 candidates. 

The first step is to find the number of mistakes by pure chance 
selection. This is easily done since we know that, among the first 
100 drawn from the hat, the number of incorrect awards will be 

X 100 = 72-6. The number of mistakes will therefore be 145. 

If, now, the scholarship e.xamination gives a correlation of -5 
with success, we find from the graph that the number of mistakes 
will be 70 per cent, of the number by pure chance, that is, 70 per 
cent, of 145. This is loi. We shall therefore give scholarships 
to 50 pupils who should not receive them, and refuse scholarships 
to 50 pupils who deserve them. Also we know that out of the 100, 
only 50 will be correct awards. 

These figures correspond very closely with the exact results 
given in Table XL. If, therefore, a Director of Education knows 
approximately the correlation between a scholarship examination 
and success, he can use the graph to find the figures relating to his 
own particular problem, figures which will be more enlightening 
to his Committee than any statement about correlations. 

SELECTION OF PUPILS FIT FOR SENIOR SECONDARY EDUCATION 

It is only in regard to senior secondary education that the 
problems are likely to arise in Scotland. The treatment, however, 
is quite general and can be applied to any similar type of selection. 

The essential difference between this selection and the preceding 
is that the pass-mark is not fi.xedfor us by the number of scholarships 
or the accommodation of the schools: we have the responsibility of 
fixing it so that we shall, as far as possible, admit those who will profit 
by a senior secondary course and exclude those who would not. 

For each sixe oj correlation there is a definite pass-mark at which 
the number of misfits is a minimum. This, as we shall show presently, 
can be demonstrated by a general proof, but for the sake of the 
non-mathematical reader we use a simpler approach. 

For the purpose of our explanation we refer to fig. 26.^ Let 
us suppose that 16 per cent, of the applicants are unfit for secondary 
education. The volume CAD will then be 16 per cent, of the 
total volume. If the pass-mark were fixed at AB, the number of 

» P. 122. 
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admit-fails would be the volume CEA, and the number of reject- 
successes would be the volume BED. The sum of these two 
volumes would be the number of misfits. 

If we know where AB is fixed we have all the data necessary 
for the calculation of these volumes. We can, therefore, for any 
size of correlation, make experiments to ascertain the effect of 
raising or lowering the pass-mark on the number and the com¬ 
position of the misfits. Table XLI is an example. 

TABLE XLI 


Effect of changing the pass-mark on the percentage of misfits 

for normal correlation 


Pass-mark at 

Failed 

Admit-fail 

Reject-success 

Total misfits 

Mean * 

500 

3-1 

37-3 

404 

- * 5 ^ 

30-9 

61 

2 I-I 

27-2 

- 1*0 

15-9 

9-6 

9-6 

19-2 

- 1-5 

6-7 

12*6 

3-4 

16-1 

-1-8 

3-6 

13-9 

1-6 

15-6 

- 1-9 

2-9 

14-25 

1-25 

15-50 

- 20 

2-3 

* 4-54 

•95 

15-48 

-2*1 

1-8 

14-79 

.71 

15-50 

-2-6 

•5 

* 5-5 

•I 

15-7 

— 00 

00 

15-9 

0*0 

1 

15-9 


f =+-5 Number of unfits = i 6 per cent. 

* In this chapter cr denotes the standard deviation of the Qualifying scores of 
the applicants for admission to senior secondary courses. 


A good many of the principles of selection are concentrated in 
this Table. When we cut off i6 per cent, of a normal distribution 
the cut is made at approximately la from the mean. OF = A is, 
therefore, in this case -Ict. If we draw the pass-mark AB also 
at - icr, AB and CD would meet on the major axis of the elhpse, 
and the position would be the same as for the award of scholar¬ 
ships. The number of admit-fails should be the same as the 
number of reject-successes, and this is so in the Table. 

Studying the figures in the last column we see that the per¬ 
centage of misfits is a minimum when the pass-mark is drawn at 
- 2c. If it is drawn above that point the number rises steeply 
and considerably; below that point the number again rises, but 

much less steeply. 
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It is interesting to find that the pass-mark for minimum misfits 
is so much lower than the line separating successes from failures. 
The latter is i<7 below the mean, and cuts off 16 per cent, of the 
pupils; the former is 2a below the mean, and this would cut off 
less than 3 per cent. 

An important practical point is involved here. Even if we 
knew the number of fit applicants this would not enable us to fix 
the pass-mark. If we fixed it in such a way as to admit the pro¬ 
portion we knew to be fit, it would be much too high, and the 
number of misfits would be well above the minimum. In certain 
cases, if we wish the misfits to be a minimum we must admit far 
more than the number who are actually fit. 

We note, too, that in this particular case the number of admit- 
fails at the minimum greatly exceeds the number of reject-successes. 


the pass-mark at which the number of misfits is a minimum^ a 
pupil has a 50 pet cent, chance of success in the secondary course; 
below this pass-mark the pupil has more than a 50 per cent, chance 

of being a failure. This can be demonstrated in a very simple 
way from fig. 31.' 

Let CD represent the line dividing successes in the secondary 

course from failures, and let RS be the line joining the two 

horizontal tangents to the ellipse. RS passes through the centre 

of the ellipse, and on it He the mid-points of all horizontal chords. 

Consider now the point E where RS cuts CD. If we draw through 

E a horizontal chord AEB, AE will be equal to EB, and the volume 

of the sohd on a narrow strip cut at AB wiU be divided into two 

equal parts at E. This means that, at AB, half the pupils are 

successes m the secondary course and half are failures. That is to 

say, a pupil with the entrance score corresponding to AB has just 

an even chance of success in the secondary course. With this 

pass-mark ^e number of misfits will be the sum of the volume 
LEA and BED. 


If we raise AB sHghtly (E being the point at which AB cuts CD) 
we Shall reduce the volume CEA and increase the volume BEd! 
But since, m the raised position, AE will be shorter than EB, the 

than'^fh reject-successes tviU be greater 

than the amount by which we reduce the admit-fails. There- 

Dresenr Lc-f ^^^ow its 

present position we again increase the number of misfits. AB is 

' P. 132. 
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thus the point at which the misfits are a minimum, and it is the 
point at which the pupil has just an even chance of success.^ 



SUCCESS JN SECONDARY SCHOOL 
Fig. 31.—DeterminatioD of correct pau-mark. 

We shall as before call this point at which the number of misfits 
is a minimum the correct pass-mark. 


Curves of Misfits 

If we know the correlation between an entrance test and success, 
we can, for any proportion of unfits in the group of candidates, 
calculate the number of misfits for any pass-mark. Thus we can 
plot a graph which will show in a clearer way the effect of raising 
or lowering the pass-mark. 

We can also do this for different sizes of correlation, and see how 


the curves compare with one another. 

Such curves are given in fig. 32, a diagram which should be 
studied by all who have to deal with selection. It is based on a 
proportion of fits to unfits which is very close to that found in the 

1 The minimum is always at the point where the regression line of the 
x-arrays cuts the line dividing successes from failures. For the mathe¬ 
matician. who has read the above popular demonstration with impatience, it 
may be added that a proper mathematical proofis easy We take 
for the sum of the volume CEA and BED as given in footnote p. 123, diffMcn 
^te andTq^te to aero. Mr A. S. Robertson, who earned out the operation, 
found it to be much less difficult than might be expected. 




PERCENTAGE OF MISFITS 
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City in which the Inquiry was conducted, and is likely to be 
reasonably similar to that in other parts of Scotland. We take it 
that about 72-6 per cent, of the applicants are fit for secondary 
education and 27-4 per cent, are unfit. Curves for four sizes of 
correlation are shown. 



PASS MARK (cr) 

Fic. 32.—C^vM of mi$6t$ for different lizes of correlation. 
(Proportion of unfit lew than half.) 


If thl pass-mark is raisrd abovr thr comet point the number of misfits 
rises to a mascimum equal to the number of jit applicants: if lowered 
bel^ the correct point, a maximum equal to the numberwhoare unfit. 
This IS obvious with or without the help of fig. 32. 
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If the number of Jit applicants is greater than the number of unfits 
the number of misfits rises more quickly when the pass-mark is raised 
above the correct point than when it is lowered below it. This principle 
is clear from a study of the curves in fig. 32, and it will be seen 
that the effect is more pronounced with the lower correlations. 
It is an important one, for it shows that the danger, in Scottish 



Pio. —Curvet of misfitt for different lizei of correlation. 

(Proportion of unfit one-half*) 


conditions, lies on the side of a high pass-mark rather than on the 

side of a low one. . 

We here come upon a curious and somewhat annoying tact. 

Nearly all the laws depend upon the point where the vertical CD 

in fig. 31 cuts the distribution, that is, upon the proportion 

of fit to unfit candidates. Generally speaking, if J certain law 

holds when the proportion of unfit is 1^^*' 

reversed if the proportion of unfit is more than half. The pr«ent 

principle is a good example. Fig. 32 ^ 

Ae pLportion of unfit is less than hah-. F.g. 33 *hows the 

situation when half are fit and half are nf misfits 

The curves now become symmetncal, and the number of misfits 

rises eauaUy steeply on both sides of the correct pass-marks. 

U n^w the percLtage of fits were less than half, say zr^ per 
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cent., the curves would be the mirror images of those in fig. 32, 
and the misfits would rise more steeply on the side of lowering 
the pass-mark. 

This brings out the important fact that, in considering the 
problems which arise in selective examinations, such as the fixing 
of pass-marks, the Education Committee must adapt its procedure 
to the percentage of unfits in the group of candidates. There is, 
of course, no force in the objection that this could not be known 
unless all the candidates went on to the secondary course; and 
that, even then, the information would be available three years 
too late. All that is needed is a general idea of the proportion, 
and every Director of Education is in a position to know this. 
In Scotland, for example, the proportion of unfit candidates for 
secondary education will almost certainly always be less than 50 
per cent. In the City in which the Inquiry was made it is about 
29 per cent., and varies little from year to year. Fig. 32 is there¬ 
fore the case that is relevant to Scottish conditions. 

The practical value of an entrance examination depends not 
merely upon its correlation with later success but also upon the 
percentage of unfit candidates. By practical value we mean the 
value to the Education Committee in ensuring that the number 
of misfits is a minimum. 

The truth of the principle can be seen by a comparison of 
figs. 32 and 33, from which we see that an examination giving 
a correlation of -5 with success might be of some use if half the 
candidates were unfit, but that it would be almost useless if only 
27 per cent, were unfit. 

If the percentage of unfit candidates is under 30 it is probabU 
that many of the present entrance examinations are a waste 
of time and money. The principle is stated in a provocative 
way, but the following are the facts upon which it is based. We 
have already pointed out that the average correlation between 
entrance examination and success recorded in four published 
investigations was -34; and, judging from the data available, 

we doubt whether most entrance examinations have a correlation 
as high as *4. 

If that is not so, our principle has to be modified. But, taking 
first an examination with a correlation of *5, we see from fig. 32 
that the minimum misfits, if we used the correct pass-mark, 
would be 25 per cent. If we used no examination at all, and 
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admitted all the candidates, the number would be 27*43 per cent.: 
the saving of misfits would thus not be very great, even if we 
managed to hit the exact pass-mark. But if, as in our experience 
very often happens, the pass-mark were too high, say -*517 
instead of - i* 2 < 7 , the number of misfits would be 28*34 cent.; 
that is, more than we should have if we admitted all the candidates. 

If the correlation were +*4, the minimum misfits with the 
examination would be 26*55 cent.; with a correlation of 


+ *3 it would be 27* 
This does not n 


23 per cent.^ 


give up its entrance examination—merely that it should take 
steps to find out the correlation which the examination gives 
with success, and the normal percentage of unfits among its 
candidates. If the results, considered in the light of what has 
been said above, show that the examination is nearly valueless, 
it should face the facts and mend its ways. This can be done 
by improving the setting, administration and correction of the 
examination, and by adopting some of our later recommendations 
as to the inclusion of other measures in the battery. 

We leave as an exercise to the reader the consideration of the 
case where the percentage of fit candidates is 30 pc^ cent, or less 
and the correlation is about *4, hoping that he will not conclude 
that the best plan would be to reject the lot—thus saving not 
only the cost of the examination but also that of the secondary 
education. 


Ij the number of fit candidates is greater than the number of unfit, 
then the higher the correlation between the entrance examination 
and success the higher is the correct pass-mark. This important 
result is clear from fig. 32. The point at which we get the 
minimum of misfits is lower for the lower correlations, the law 
of fall being indicated by the broken curve. The difference is 
not very great for correlations between *75 aiid i*o, but below *5 

the correct pass-mark falls very rapidly. 

As usual we find the neutral point when 50 per cent, are unht. 
The correct pass-mark is then always the same, as can be seen 
from fig. 33. If the percentage of unfit pupUs is more than 50 


. I„ Scotland the distribut^n “f rdtolbutit the 

°n[rancrexaminat“i 7 n ^Zd L of tomewhat greater value, 'h' 

for correlation, of + -4 and + -3 being about 25-5 per cent, and 27-0 per cent. 

respectively. 
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the law is reversed, and the higher the correlation the lower 

should be the pass-mark. , , . , u 

In fig. 34 we give the exact curve of fall for the case where 

27-4 per cent, of the candidates are unfit, and from this a Director 

of Education can get a rough idea as to where his pass-mark 

should be drawn if he can estimate the size of the correlation 

between entrance test and success. 



If the number of fit candidates is greater than the number of 
unfit, the pass-mark should never be fixed above the point at which 
it will cut off a number equal to the number of unfit. It can easily 
be demonstrated that if the line separating the fits from the 
unfits is at -per, then the pass-mark will be highest when r = i*o, 
and it will then be at -pa. 

The rule may serve as a rough guide to a Director of Education. 
If he estimates, from the experience of years, that the percentage 
of unfit candidates is about 27, he knows that the pass-mark 

5 * 
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should not be above —■6(7. AH he has to do, therefore, is to 
find the standard deviation of his entrance examination marks, 
multiply it by *6, and subtract from the mean. The pass-mark 
should certainly not be above this point, unless the distribution 
differs considerably from the normal form. 

^he higher the correlation between, entrance examination and 
success^ the more critical is the pass-mark and the more important 
it ts that it should be correctly fixed. The importance of the 
correct fixing of the pass-mark, in this type of selection, has 
perhaps not yet been sufficiently realised; and in this principle 
we draw attention to the fact that the better the entrance test 
the greater is the danger. This is at once clear from the curves 
in figs. 32 and 33. If we take an allowable error in fixing the 
pass-mark to be that giving an increase in the misfits equal 
to I per cent, of the candidates, then with a correlation of ro 
our latitude in drawing the pass-line in fig. 32 would be only 
± *050. With a correlation of 75 it would be ± *450. 

Even with a perfect test (that iSy giving a correlation of J*o with 
success) we could have a large number of misfits if the pass-mark 
were incorrectly fixed. This is not an independent principle, but 
its importance is sufficient to warrant separate statement. In 
striving to improve our selection it is not sufficient to perfect the 
tests. As we have seen above, a very good test is in some ways 
a positive danger. 

We shall later make some suggestions as to ways in which the 
dangers may be avoided; but in the meantime we give two 
numerical illustrations of their magnitude: 

If, with a test giving a correlation of I’O, we fix the pass-mark 
8 per cent, too high (assuming an ordinary distribution of 
examination marks), we should have the same number of 
misfits as with a test giving a correlation of 75, provided 
that the correct pass-mark were used. 

If, in our own senior secondary group of 416, with our best 
battery (giving a correlation of *8), we had fixed the pass- 
mark 10 per cent, too high, we should have increased the 
misfits by 28. This is equivalent to using a test with a 
correlation of +*57 and the correct pass-mark* 

If the number of fit candidates is greater than the number of 
unfity then at the correct pass-mark the number of admtt-fatls 
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will be greater than the number of reject-successes. The purport 
of this principle is that, if an Education Committee adopts our 
suggestion as to the correct pass-mark, it is not erring on the 
side of refusing to give the child his chance. 

The further the percentage of unfit candidates falls below 50 
the greater will be the preponderance of admit-fails over reject- 
successes at the correct pass-mark. The size of this effect, for a 
correlation of +-50, is shown below: 


Percentage 
of unfit 

Percentage .it the correct p.iss- 

•mark 

candidates 

Admit-fails Reject-successes 

MhBts 

50-0 

167 167 

33-3 

308 

21 I 6-1 

27-2 

15-9 

14-5 -9 

» 5-5 

II -5 

11 0 3 

11-4 


If the number of fit candidates is greater than the number of unfits 
the lower the correlation between the entrance test and success., the 
greater is the preponderance of admit-fails over reject-successes at 
the correct pass-mark. In fig. 35 we have plotted the curves 
of admit-fails and reject-successes at the correct pass-marks for 
different sizes of correlation. The curves are in respect of the 
case where 27*4 per cent, of the candidates are unfit, which is 
nearly the normal proportion in the City in which the Inquiry 
was made. 

This shows that, under Scottish conditions, the poorer the 
entrance test the more we err on the side of giving the pupil his 
chance, provided we use the correct pass-mark. 

If the number of fit candidates is greater than the number of unfits 
the lower the correlation between the entrance test and success the 
greater the number of pupils we have to admit. This is an important 
practical law, for it has financial implications. We assume, of 
course, that the Education Committee desires to have the misfits 
a minimum; and, if so, it is clear that it is bad economy to save 
money on the entrance testing. 

Taking what we have regarded as a fairly normal percentage 
of unfit candidates for Scotland, that is 27*4 per cent., the per¬ 
centage of candidates that we should have to admit for different 
sizes of correlation between entrance test and success is as follows: 
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Correlation 

10 

•9 

•8 

7 

•6 

•5 

•4 

•3 

•2 

•I 

0*0 


SECONDARY EDUCATION 

Percentage to be admitted 

72-6 

74-9 

77-3 

80-5 

84-1 

88-5 

93-3 

977 

99-9 

1000 

1000 



Fig. 35.—Curves showing relative proportions of admit-fails and reject-successes 
at the correct pass-mark for different sites of correlation. (Percentage of un t 27 41 - 

From this it is clear that, if we are to deal fairly with the 
pupils and make the most economical use of the national resources 
of money and talent, we must admit to the senior secondary 
schools more pupils than the number who are fit. “ ® ^ 

perfect test and used the correct pass-mark, we should admit 72- 
per cent.; with our own best battery (r = *8), we should have to 
admit 777 per cent. With an examination, giving a correlation 
of about -4, we should have to admit 93*3 per cent. If there were 
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1,000 candidates this would mean that we should have to provide 
secondary education for 160 more pupils, a large proportion of 
whom would be failures. And it will be clear by now that an 
Education Committee cannot counteract the effects of a poor 
examination by using a higher pass-mark than the correct one for 
that examination. 

If about 2y per cent, of the candidates are unfits and if the number 
of failures in the secondary course is less than 9 per cent, of the 
number admitted^ the system of selection is unsound. This is merely 
a rough rule for Scottish conditions, and it draws attention to 
an erroneous opinion that is sometimes held. Some Education 
Committees, in experimenting with different systems of selection, 
collect statistics as to the success or failure of the pupils admitted 
to the senior secondary schools; and in discriminating between 
such systems it is occasionally assumed that the one producing the 
fewest failures in the secondary school is the best. This is far 
from the truth. The fact that system A admits fewer failures 
than system B probably means that A is the less sound system; but 
the position is that we cannot judge on this basis alone. We must 
take into account the correlation, the proportion of unfit candidates 
and the position of the pass-mark. 

We have already established that it is vain to aspire to a system 
with no misfits. The question then arises as to what should be 
the percentage of failures among the pupils admitted if the system 
is as sound as present conditions permit. Our answer depends on 
the percentage of unfit candidates; but, taking this to be approxi¬ 
mately 27 per cent., and assuming normal correlation, there should 
be 14 per cent, of failures among those admitted. If the dis¬ 
tribution had a tail of weak pupils, as in our Inquiry group, 11*5 per 
cent.^ of those admitted should turn out failures. These figures 
are based on our best battery; and since the minimum number of 
admit-fails would be greater with a poorer one, it will be seen that 
our figure of 9 per cent, is conservative. 

Curve showing Predictive Significance of Correlations in the Selection 
of pupils for Secondary Education 

The nature of the curve depends on the proportion of unfit 
candidates; but if we take this to be 27-4 per cent, of the 
total, we obtain a result applicable with reasonable accuracy 

' If we used the additional principles mentioned in Chapter XIV the 6cure 
would be lo-i per cent. See also Chapter XVII, p. 173. 
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to Scottish conditions. Fig. 36 gives the minimum number of 
misfits for each size of correlation as a percentage of the minimum 
number for zero correlation; that is, it assumes that the correct 
pass-mark is always used. 



SIZE OF CORRELATION 

Fio. 36.—Signlficaoce of differeat size* of correlation in lelcction of pupil* fit 

for secondary education. 


Note. —^The general laws stated in this chapter will hold for 
all distributions likely to be met in practice,_ but numbers and 
percentages based on the assumption of normality should be used 
only as rough guides. Fig. 36, for example, gives the numbers 
of misfits for different sizes of correlation if the distribution ot 
candidates is normal. We think it probable that, in Scotland, 
the distributions will usually have the tailed form of our senior 
secondary group, and, if so, the numbers would be as foUows: 


Correlation 


o 

I 

z 


3 

4 

5 

6 

7 

8 


Minimum number of misfits 

100 

100 

100 

99 

• 96 

87 

75 

61 

5 * 

35 

o 


•9 

10 




CHAPTER XV 


ATTEMPT TO IMPROVE SELECTION BY SPECIAL 
CONSIDERATION OF BORDER-LINE CASES 

So far we have assumed a hard-and-fast pass-line. We now 
consider whether a careful scrutiny of border-line cases might 
justify the admission of certain pupils just below the pass-mark or 
the rejection of some just above it, thus improving our selection. 

One advantage of the misfits technique is that we can make 
experiments on such problems with an ease and speed much 
appreciated after the laborious correlation procedure. 


Initial Standard Number of Misfits 

For the purpose of the experiments which follow we use the 
complete battery IQ + Q + S + Ts, for which the minimum misfits 
in the senior secondary group are as follows: 

Number in group Admit-fails Reject-successes Misfits 

452 39 20 59 

These, then, are our standard figures in relation to which we 
assess the value of any proposed principle. 


Supplementary Principles involving Use of Test 

AND Examination Scores 

imposing a minimum in each subject for all pupils 

The subjects are intelligence, English and Arithmetic. English 
is taken to be Qe-i-E + Tes, and Arithmetic Qa + A+Tas. By 
imposing a minimum of o sigma we mean that a pupil is rejected 
if he falls below o sigma in any subject, no matter how high his 
average score may be. 
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We give below the numbers of misfits with different minima: 


Minimum 

Admit-fails 

Reject-successes 

Misfits 

+ -6 sigma 

20 

62 

82 

+ '4 

26 

33 

59 

+ -2 

32 

26 

58 

00 

34 

22 

56 

- -2 

38 

21 

59 

- -4 

39 

20 

59 

- *6 

39 

20 

59 

Standard figures 

39 

20 

59 


In interpreting this and the following figures we have to bear 
in mind the difficulty that, while the complete group is reason¬ 
ably large, the numbers affected by a principle may be small. 
There is no satisfactory escape from this; but we can introduce 
some of the steadiness which comes from the numbers above and 
below by using graphs. By examination of the sweep of these, 
after smoothing, we can usually tell whether or not a critical 
figure is likely to be accidental. In the above case when we 
plot misfits against the minima we find confirmation that there 
is likely to be a minimum at o sigma, but not quite so low 
as 56. 

With this general reservation we hazard the following con¬ 
clusions: 

A small reduction in the number of misfits can be made by 
imposing a minimum in the separate subjects. 

The minimum has to be very exactly determined: if it is too 
low it gives no improvement; if it is too high the number 
of misfits is greatly increased. 

The practical import is that the principle should be used only 
if the minimum is scientifically determined by a method which 
we shall explain later. 


IMPOSING A MINIMUM IN EACH SUBJECT FOR WEAKER 

PUPILS ONLY 

To impose the minimum only on pupils with average scores of 
+ 1*0 sigma and under is the same as to demand a higher 
pass-mark, namely, +l-o sigma, if the pupil falls below the 
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minimum in any part of the examination. The numbers of 
misfits \vith different minima are shown below: 


Minimum 

On pupils below 

Admit- 

faUs 

Reject- 

successes 

Misfits 

0*0 sigma 

+ 1*5 sigma 

34 

22 

56 

+ IO 

35 

22 

57 


+ -9 

37 

22 

59 


+ -8 

37 

21 



On all pupils 

34 

22 

56 

+ *2 sigma 

+ 1-0 

34 

^4 

58 

+ -8 

36 

22 

58 


On all pupils 

32 

26 

1 ' 
1 00 

+ *3 sigma 

+ 1*0 

31 

26 

57 


+ -8 

H 

23 

57 


On all pupils 

31 

28 

59 


Clearly nothing is gained by limiting the application of the 
minimum to the weaker pupils. 


IMPOSING A MINIMUM (< 2 ) IN IQ ONLY, (^) IN ENGLISH AND 
ARITHMETIC SEPARATELY BUT NOT IN IQ 

We found that both of these gave poorer results than the 
imposition of a minimum in each subject as in the preceding 
sections. 


ALLOWING A LOWER PASS-MARK IF THE PUPIL HAS 
HIGH ABILITY IN ONE SUBJECT 

This is the converse type of allowance. It is based on the 
idea that a pupil who falls just below the pass-mark but has a 
very high IQ or very high ability in English or Arithmetic, may 
have a better chance of success in the secondary course than a 
pupil who is just above the border-line in all subjects. The 
-number of misfits when a lower pass-mark is allowed for pupils 
with high ability in any subject is as follows: 
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Test of high 
ability 

Lower 

pass-mark 

allowed 

Admit- 

fails 

Reject- 

successes 

Misfits 

Over +1 -3 sigma 

+ •6 sigma 

39 

19 

58 


+ •5 

40 

19 

59 


+ •4 

*9 

41 

60 


+ •3 

19 

4 * 

60 

Over +!•! sigma 

+ •6 

19 

42 

61 


+ ■5 

18 

43 

61 


+ •4 

18 

45 

63 


+ ■3 

18 

45 

63 

Standard figures 


39 

20 

59 


It is therefore most unlikely that any great improvement could 
be made in this way. Our results show a slight reduction if we 
allow a pass-mark *1 sigma lower for pupils who are above 
+ 1*3 sigma in any one subject. 


APPLYING BOTH (a) A MINIMUM IN ANY SUBJECT, AND (b) 
A LOWER PASS-MARK IF THE PUPIL HAS HIGH ABILITY IN 
ONE SUBJECT 

These supplementary principles may conflict when combined. 
Thus a pupil whom we have passed on account of high ability 
in English may fail because he is below the minimum in Arithmetic. 
We have therefore to fix a definite order of application, and we 
found the following to be the best: 

(1) Reject all pupils who fall below o sigma in one subject. 

(2) Admit pupils between + *6 sigma and + *7 sigma if they 

are above +‘l‘3 sigma in any subject, and are not 
rejected by reason of (i). 

By this procedure we find that the following are the numbers 
of misfits: 

Admit-fails Reject-successes Misfits 

34 21 55 

The resulting improvement (4 misfits out of 59) considerable; 
and the above now become our standard figures for the application 
of further principles. 
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PROTECTING THE PUPIL AGAINST A FLOP IN ONE TEST 

OR EXAMINATION 

We have given an example of this in fig. 19, where it is 
clear that some accidental disaster has overtaken the pupil m 
the Arithmetic examination. In diagnosing a flop it is desirable 
that we should have at least two other reliable measures tn the 
same subject', and the drop should be greater than those that 
could be explained by normal fluctuations of performance and 

marking. , , 

We cannot apply this principle to a low IQ as we have no 

other tests in the same subject to act as a standard. Yet with 

many pupils we had the strongest suspicion that the IQ result 

was of the flop type, and that this increased the number of misfits. 

We therefore make two suggestions. It is desirable that further 

research be undertaken towards obtaining comparable IQs from 

different group tests. It would then be an advantage if Education 

Committees were to arrange for the application of a group or 

individual intelligence test to all primary pupils at least a year 

before the Qualifying stage. The results of such testing could 

be used to protect the child against the danger to which we have 

referred, and could be put to many other uses. Another method 

would be to apply an individual test in all suspected cases. 

After some experiment it was found that the best system was 
to count a score a flop if: 

(1) The other two scores in the same subject were within 

•3 sigma of each other. 

(2) The third score was at least 7 sigma below the lower of 

the other two. 

The effect of protecting against a flop in one test is shown 
below: 

Admit-fails Reject-successes Misfits 

After allowance for flops 34 19 53 

Standard figures . . 34 21 55 

The principle thus makes a reduction of 2 misfits in 55; and, 
while the numbers are so small that we cannot vouch for the 
result with any great degree of statistical assurance, our experience 
of several years of practical selection warrants the recommendation 
that it should be accepted. 

It will be understood that the number of flops in our group 
was much greater than 2. The difficulty, indeed, with all these 
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subsidiary principles is that they are two-edged weapons. A 
number of pupils who were satisfactorily housed^th the reject- 
fail group, and whom we should have been only too glad to leave 
^ere, were transferred by our flop principle to the admit-fails. 
They thus increased the number of misfits. 

Supplementary Principles involving Use of 

Other Data 

We now consider whether we could improve our selection by 
going beyond the test and examination results and taking into 
account such factors as health, industry, home conditions and 
personal qualities. 

ALLOWANCE FOR HEALTH 

It is unlikely that anyone, outside ultra-scientific circles, would 
think of including health ratings in the scores before the pass-mark 
is determined. The most that we could do would be to take 
it into account in extreme gradings or border-line cases, and one 
method would be to reject all who have a very low health grading, 
irrespective of what their average score may be. To test this we 
tried rejecting all with a health grading of E, and found that we 
increased the misfits by i; when we rejected all with D or E 
the increase was 15. 

Another method would be to take health into account in 
border-line cases only. If a pupil were just below the pass-mark 
but had a health grading of A, there would be a case for admitting 
him in preference to one who was just above the line but had a 
health grading of E. To test this idea we used a principle similar 
to that adopted in allotting age-allowances. We gave a bonus 
of *2 sigma for an A, •! sigma for a B, a reduction of 
•I sigma for a D, of *2 sigma for an E. This amounts to 
allowing a slightly higher or lower pass-mark according to the 
health grading. The effect of different allowances for health on 
the number of misfits is shown below; 


Health allowance 

Admit-fails 

Reject-successes 

Misfits 

A+ '4 : B + '2, etc. 

43 

19 

62 

A + *2 : B + -I, etc. 

37 

19 

56 

A + -I : B + -05, etc. . 

37 

19 

56 

A+ 02: B + -OI, etc. . 

35 

>9 

54 

Standard figures . 

34 

19 

53 



BORDER-LINE CASES H 9 

The conclusion is only too obvious, and is confirmed by the 
method mentioned on p. 144. If we plot number of misfits 
against size of health allowance the curve reaches the minimum 
when the health allowance is zero. 


ALLOWANCE FOR INDUSTRY 



Fic. 37*—Case where allowance for industry, etc.> would lead to error. 


is +*70, and this girl’s average score is +‘69; yet she has very 
pronounced practical ability; is well above the average in industry 
and in health; has originality and special talents in Art and Music; 
comes from a good home; and her primary head master is 
confident that she will be a success. Surely we were intended 
to admit this girl and give her the chance of a senior secondary 
education which may be the road to a career in music or art. 
We should in this case have been wrong. 

fig- 38 we give the opposite type of case, where we should 
have been saved from error by taking into account the teacher’s 
forecast and industry grading. 
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As with health, we first rejected all pupils with an E in industry. 
This made no change in the number or composition of the misfits. 
Rejection of all with D or E ratings left the total number of 
misfits unchanged, but slightly lowered the number of admit- 
fails relative to the reject-successes. We also tried the method 
of assigning a bonus or penalty in accordance with the grading, 



Fig. 38.—Case where allowance for indujtrjr, etc., would improve lelection. 


thus taking industry into account only in border-line cases. 
effect of such allowances for industry on the number of misfits 
is shown below: 


Industry allowance 

Admit-fails 

Reject-successes 

Misfits 

A + i o : B + -5, etc. . 

49 

12 

61 

A + *8 : B + 4, etc. . 

45 

12 

57 

A+ 6 : B + -3, etc. . 

43 

H 

57 

A+ 4 : B + -2, etc. . 

41 

H 

16 

18 

>9 

55 

56 

57 

55 

A+ 2 ; B+ i, etc. . 
A+ -I : B + -05, etc. . 
A+ *02 : B+ oi, etc. . 

40 

39 

36 

Standard figures . 

34 

<9 

53 
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We can only conclude that nothing is to be gained, even 
in border-line cases, by taking account of industry gradings as 
obtained in our Inquiry. Perhaps one reason for this is that 
the pupil’s industry will be reflected in his test and examination 
results and is thus already taken into account. 


ALLOWANCE FOR TEACHERS* FORECASTS 

On the rating cards the primary teachers gave forecasts of the 
pupils’ success in each of the four main types of post-primary 



Fio. 39.—Case where allowaDcc for tcacher'i forecast would lead to error. 


course. A five-point scale was used, in which A indicated 
distinguished success, B clear success, C doubtful success, and 
so on. 

We had high hopes of reducing the misfits through their use, 

for we noted frequently that pupils whose success was greater 

than we should have expected from their test results had forecasts 

of A or B. But once again we had a compensating number of 

cases, like that shown in fig. 39, where the forecast was clearly 
wrong. 
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This pupil has an average score of + 72 sigma and therefore falls 
just above the pass-mark. But her primary teacher reports that 
she is untidy, slow in speech, movement and learning, and grades 
her as a hopeless failure in a senior secondary course. Surely, 
on such additional information we should bring her below the 
line. The right-hand side of the profile shows that, had we 
done so, we should have made a decided blunder. The pupil 
took a one-language course, attained nearly the average of the 
secondary group in her first two years, and showed signs of 
improvement even on that satisfactory standing. 

By failing all pupils with a teacher’s forecast of E we increased 
the misfits by 2, while rejection of all with forecasts of D or E 
gave an increase of 4. 

Using the forecasts in border-line cases only we get the follow¬ 
ing results: 


Forecast allowance 

Admit-fails 

Reject-successes 

Misfits 

A + i'O : B + -5, etc. . 

41 

18 

59 

A+ -8 : B + -4, etc. . 

40 

22 

62 

A+ *6 : B + -3, etc. . 

40 

21 

61 

A+ 4 : B + -2, etc. . 

39 

20 

59 

A+ -2 : B + •!, etc. . 

39 

20 

59 

A 

A+ •! : B + -05, etc. . 

37 

21 

58 

A+ -02 : B + -OI, etc. . 

36 

20 

56 

Standard figures 

34 

*9 

53 


The conclusion is that the use of the teachers’ forecasts even 
in border-line cases increases the number of misfits. 


ALLOWANCE FOR PERSONAL QUALITIES 

Primary teachers were asked to note any marked personal 

qualities of the pupils. This was not a YpL a 

coUecting the data, but we felt that it would be sufficient for 

preliminary exploration of the problem. nf 

^ The view has often been expressed that existing systems of 
selection are based too exclusively upon intellect and scholastic 

attainment, and that success in higher education f 

not merely upon these but also upon certain person qu hti«^ 
At first sight our case histories seemed to bear tffis out. In 

reviewing the profiles we encountered a number ® . 

j ul^undfr the admission fine had a number of good quahties and 
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proved a success; and conversely where a pupil just above it 
had marked bad qualities and was a failure. The matter was 

therefore worthy of further study. ^ ^ 

To reduce the results to a form amenable to statistical treatment 

was far from easy, but the following rough procedure was finally 
adopted. We divided the qualities into three categories, good, 
bad and neutral. In so doing we were guided largely by common 
sense, and partly by the results of previous studies, which suggest 
that the most important qualities from this point of view are 
those like logical and critical ability, independence of thought, 
eagerness to learn, and are not to be found on the side of emotional 
life and character.* In the good category we included such 
qualities as perseverance, reliability, earnestness, ambition; in the 
bad group, carelessness, indifference, laziness, ready discourage¬ 
ment, and so on. To the neutral category we relegated all 
those that seemed to be irrelevant to success in the secondary 
course, for example, amiability, distinct charm of manner, good 
nature, sensitiveness, timidity. In order to take the qualities 
into account in border-line cases we gave a bonus for each good 
quality and a penalty for each bad one. The effect of these allow¬ 
ances on the number of misfits is shown below: 


Bonus or penalty for 
personal qualities 

Admit-fails 

Reject-successes 

Misfits 

*3 sigma 

39 

18 

57 

•2 

38 

20 

58 

•I 

39 

>9 

58 

•OS * 

37 

18 

55 

Standard figures 

34 

*9 

53 


Once again we get the disappointing result that, however well 
the idea may work in individual cases, it is not a success when 
generally applied. If, therefore, personal qualities are to be used 
in selection, the data must be obtained by a better method than 
that used by us. 

There is, however, a reason why the use of personal qualities 
in selection is unlikely to make any great improvement; as with 
industry, the qualities which matter are already reflected in the 
pupil’s test and examination results. 


* O, Bobertag, ScbuUrauslese. Berlin, *95+* P' ^4' Privately printed for 
the International Examination Inquiry. 
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ALLOWANCE FOR HOME CONDITIONS 

By this time we had abandoned all hope of improvement 
through the use of such supplementary data, but for the sake of 
completeness we felt we should not leave this last stone unturned. 
We divided the reports on home conditions into good, bad and 
neutral, and examined the effect of taking these into account at 
the border-line: 


Bonus or penalty for 
home conditions 

Admit-fails 

Reject-successes 

Misfits 

*4 sigma 

44 

IS 

59 

•3 

40 

17 

57 

•2 

41 

18 

59 

•I 

37 

18 

55 

•05 

37 

19 

56 

Standard figures 

34 

19 

53 


Despite this negative result, which may be due to our method 
of collecting the data, we feel that home conditions do have an 
important bearing upon a child’s success in the secondary school, 
and we shall return to the point at a later stage. 


ALLOWANCE FOR COMBINATIONS OF THE SUPPLEMENTARY DATA 

It is just possible that, while we secure no improvement by 
taking the additional factors separately, we might do so by taking 
them together. One pupil might, for instance, survive with an E 
in industry, another wdth an E in health, another with extremely 
bad home conditions. But if the same border-line pupil had all 
three we might be justified in prophesying that he would be a 
failure. 

We tried various combinations without any resulting improve¬ 
ment; and finally we included every rating. For each measure 
the allowance made was that giving the least unsatisfactory result 
when it was taken separately, and an age allowance was added. 
The result was that the total number of misfits was increased from 
the standard figure of 53 to 61. 




border-line cases 
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MORE SCIENTIFIC SYSTEMS OF MAKING ALLOWANCES 

So far we have used the straightforward system of giving a bonus 
for an A or a B and a penalty for a D or an E. This is probably 
the only system ever Ukely to be used in practice, but it has a 
defect which may be indicated by the following Hgure: 

E D CHF B a 

-2-20 sigma - roy o +1-07 +2-20 

p,G. _The need for a more acieotific ayitem of making aUowancci. 


On the assumption that the industry gradings are normally 
distributed, the means of the categories A, B, C, D and E would 
be as shown. If, at C, the pupil had a 50 per cent, chance of 
success, the system which we have used would be sound; but if 
the point for an even chance of success were at F (that is, if a pupil 
had to be more industrious than the average to have an even 
chance of success in the secondary school), a C grading should get a 
penalty, and the bonus for a B would be less than the penalty for a 
D. A more scientific system would therefore make the bonuses 
and penalties proportional to the distances of the means of the 
categories from F. 

Industry 

The data determining the point at which the pupil has an even 
chance of success have some interest in themselves, and are given 
below: 


Industry grading 

Successes 

Failures 

Total 

Percentage 

failures 

A 


2 

54 

370 

B 

192 

29 

221 

13-12 

C 

73 

76 

149 

51-01 

D 

2 

20 

22 

90-91 

e 

0 

5 

5 

100-00 


There is a clear correlation bet^veen industry and success; and 
the point for an even chance comes out to be -*15 sigma. As 
the mean of C category is - *18 sigma, the straightforward system 
of bonuses is here justifiable. 
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Treachers* Forecasts 


The percentage of failures in the various categories of the 
teachers* forecasts are as follows: 


Forecast grading 

A 

B 

C 

D 

E 


Successes 

Failures 

Total 

Percentage 

failures 

52 

2 

54 

370 

214 

30 

244 

12-30 

41 

59 

100 

5900 

8 

31 

39 

79-49 

2 

10 

12 

83-33 


There is again evidence of correlation, but it is interesting to 
note that nearly 4 per cent, of those whom the teachers expected 
to be distinguished successes have actually proved to be failures, 
and that about 17 per cent, of those for whom they predicted 
hopeless failure have been successful. By the time the reader has 
finished this Report he will, we trust, fully realise that this is no 
reflection upon the care or judgment of the teachers. 

At this point we interpolate a Table prepared by Mr Robert 
Robertson, M.A., which throws additional light upon the reliability 
of teachers* forecasts. 


TABLE XLII 


Teachers’ forecasts of success in senior secondary course for 

different groups 


Group 

N 

Percentage 

A 

B 

c 

D 

E 


1 

Senior secondary 

1 

452 

12*1 

54-6 

21-8 

8-8 

2-7 

Junior secondary 

Special classes for backward 

1,930 

•6 

17-7 

34-4 

329 

1 

14-4 

78-9 

children 

509 

0 

•2 

•9 

200 


These data show that the primary teachers can make a fair 
sifting of the material at the Qualifying stage, but that we could 










border-line cases 
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not rely solely upon their estimates. If, m the ^ 

group, we had used the teachers’ estimates alone, we should have 
Ld 42 misfits in the A, B, D and E categories; and, m addition, 

100 pupils were graded C. 

Finding that the point for an even chance is Jigma. we 
constructed two scientific bonus systems, m one of which A got 
a bonus of +-20 sigma, and in the other +'io sigma. Neither 
was of any avail, the total number of misfits remaining obstinately 
the same, though the composition was considerably altered. 


Health and Personal Qualities 

For these factors the correlation is so small that the point for an 
even chance of success cannot be determined. The percentage o 
failures in thevarious groups of personal qualities is given merely for 
its general interest to any who may contemplate a more scientific 


attack on this problem 

• 

« 




Personal qualities 

Successes 

Failures 

Total 

Percentage 

failures 

3 good qualities 

2+ 

4 

28 

H *3 

2 good qualities 

61 

5 

66 

7-6 

1 good quality . 

79 

16 

95 

lO'o 

No marked qualities . 

136 

79 

215 

367 

1 bad quality . 

18 

26 

44 

59* 

2 bad qualities . 

2 

2 

4 

500 


AGE ALLOWANCES 

We give some results bearing upon one particular aspect of this 
much-discussed topic. 

Relevance of the Age-Allowance Procedure to Scottish Conditions 

The introduction of age allowance usually springs from a desire 
to be fair to all candidates in the competition for a limited number 
of scholarships or places in secondary schools. If the examination 
is confined to pupils between 11 and 12 years, it is felt that the pupil 
of 12 has an unfair advantage over the pupil of 11. The younger 
child should therefore be assigned a bonus; and the test of fairness 
is that the percentage of awards should be uniform throughout the 
age-range. 

Morally the idea is excellent; and it is a pity that the need for it 
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should arise in this system of limited competition whose educational 
morality is open to question. Even here a wider view of fairness 
is probably necessary; and it is certainly so in Scottish conditions 
at the Qualifying stage. Our bursaries are intended to meet the 
needs of all pupils desiring a secondary education, and fitted to 
profit by it, who would be debarred by reason of the expense 
involved. Our selection system would be contrary to the statutes 
if it involved the exclusion from secondary education of pupils 
who had more than an even chance of success. So far as we have 
competition, it should be limited only by this ability to profit, not 
by a fixed number of bursaries or places in secondary schools; 
and it follows that our test of fairness must take the child*s success 
in the secondary course into account. If we admit a young pupil, 
by giving him a bonus for his youth, our fairness may be costly 
to him should he prove to be a failure. 

Ej^ect of Straightforward Age Allowances on the Number of Misfits 

We first tried the effect of the simplest form of age allowance, in 
which a bonus of so many marks is given to a pupil for each month 
by which he is below the maximum age, A common allowance of 
this type is 1*2 per cent, a month, and to assist in interpreting the 
following data it may be mentioned that *06 sigma is equivalent 
to about I per cent. The following are the effects of different age 
allowances on the number and composition of the misfits: 


Age allowance per month Admit-fails Reject-successes 

Misfits 

•10 sigma 

49 

37 

86 

•05 

42 

27 

69 

•04 

39 

24 

63 

•03 

39 

24 

63 

•02 

36 

21 

57 

•01 

37 

20 

57 

-01 

39 

20 

59 

-03 

44 

22 

66 

-■05 

56 

31 

87 

-•10 

74 

45 

119 

Standard figures 

34 

*9 

53 

To convince the reader that the result is not due to 

chance 

fluctuations of small numbers 

we 

give the graph, from which it 

will be seen that the general tendency of the curve is clear. 1 he 

negative allowances are, of course, i 

merely of the nature of a 

mathe- 
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BORDER-LINE CASES 

matical diversion. They mean that we give the old pupil a bonus 
simply for being old, and that we penalise the young pupil for his 
youth. Even with as modest an allowance as J per cent, a month 
we increase the number of misfits, and the higher the allowance the 
greater is the increa:e. An allowance of i per cent, a month would 
be almost as harmful as a negative allowance of •$ per cent. 



ACE ALLOWANCE PER MONTH 

(sigma) 

Fio. 41.—Effect of dlfferect age allowances on the number of misfits. 


Effect of more Scientific Age Allowances on the Number of Misfits 

Certain defects of the straightforward type of age allowance are 
brought out by figures i, 2 and 3,' which show that the age 
allowance for each month should not be the same for the weak 
as for the strong pupils. A scientific allowance should therefore 
depend on the pupil’s score.* 

From our data we prepared the following differential allowance: 


Pupil’s score 
Over +1*32 sigma 
+ •50 to +1-31 
- -50 to + -49 
Below - -50 


Age allowance per month 

+ 03 sigma 
+ •04 
+ •05 
+ 03 


* Pp. lO-lI. 

• G. H. Thomson, “The Standardisation of Group Tests,” Britisb Journal of 
Psychology, vol. u, Parts H and III. J. B. T. Davies and G. A. Jones, The 
SeUclion of Children for Secondary Education. London: George G. Harrap & 
Co., Ltd., 1936. Chapter vii. 
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With this system the number of misfits is increased from 
53 to 66. 

^ These results should be considered in conjunction with those 
given earlier in regard to correlations with age, the high correlation 
between intelligence score and intelligence quotient, and the close 
agreement between the follow-up correlations for these.^ 


Summary 

This has been a depressing chapter of negative results and dis¬ 
carded principles; yet it has cleared the air, and others may profit 
by our mistakes. For the benefit of future investigators we would 
emphasise one important point. A study of these supplementary 
factors in border-line cases can be most satisfactorily made if no 
selection system has been operative. In this we were especially 
fortunate, for our Inquiry deals with the last group admitted to 
senior secondary schools without a fairly strict selection. 

Despite our disappointments, we have gathered a crop of 6; and 
a saving of 6 misfits out of 59 is worth making. The correlation of 
the complete battery with success was *800; if we use the supple¬ 
mentary principles the result is equivalent to a correlation of *854. 

The only additional principles which we can recommend are: 

The imposition of a minimum for each separate subject. 

The allowance of a slightly lower pass-mark if the pupil shows 
high ability in one subject. 

The protection of the pupil against the effects of a flop in one 
subject. 

We shall show later how these principles could be used in practice. 

The extent of our success in reducing the number of misfits is 
shovni below. The group numbered 452, of whom 320 were fit 
for secondary education: 

Method of selection Number of misfits 

All applicants admitted 

Complete battery (r= +’8) with pass-mark 

of +7 sigma . . • • • 

The same, with above additional principles 


132 

59 

53 


» See Note by Professor Godfrey H. Thomson on p. 161. 
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Note by Professor Godfrey H. Thomson 

This fact, that giving age allowances increases the number of 
misfits in the senior secondary courses, is in agreement with what 
1 too have sometimes found in researches in England, namely 
that entrance examination marks without age allowances correlated 
slightly better with secondary school performance than did marks 
with age allowances. These results might appear to be an argu¬ 
ment against age allowances. But I do not draw that conclusion, 
for the object of an age allowance is not to improve prediction but to do 
justice to children born in different months of the year. 

It is true that age allowances, at any rate at first sight, seem to be 
less imperatively necessary at this stage in Scotland than in England. 
In England the examination is more competitive, and is almost 
invariably held only once a year, whereas in some Areas in Scotland 
it is held every six months. In England a pupil as a rule must sit 
the examination with his age-group, and save in exceptional 
circumstances has no further chance; in Scotland he may wait 
until a later age, and may even then be allowed to sit again if 
unsuccessful. 

But these diflFerences are more apparent than real. The 
examination in Scotland is really competitive, even if it be less so 
than in England. There is a mark below which the candidate 
does not get a bursary; a mark below which he is not recommended 
to go to a senior secondary school but rather to a junior secondary 
school; a mark below which he is not allowed to pass on to any 
Und of secondary school. The difference between England and 
Scotland is rather that the competition in Scotland is at a different 
level of ability; but it is still there. 

Not by any means in every Area in Scotland is the examination 
held every six months; and even where it is, one of the examina- 
tions usually has many more candidates than the other. And 
although there may be another examination only six months ahead 
a boy who has been recommended to go to a junior secondary 

school may take the advice, and not try again. Indeed he is not 
always allowed to do so. 


6 
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I feel strongly then that age allowances should be given, at least 
over the six or twelve months -within which the main body of 
candidates were born, and a reasonable period beyond that with 
certain precautions. If this is not done, then the younger children 
born within that period of six or twelve months win bursaries, 
or will get into secondary schools, in smaller numbers than the 
older ones, and this is obviously unfair. If it be replied that these 
young candidates have another chance next year, or even in six 
months’ time, I have already answered that in part; and in any 
case failure is discouraging. But even if they try again, they are 
still at a disadvantage if no age allowances are given; for among 
the group of children who are sitting a second time, these are still 
the youngest. 

I admit that it is very difficult in Scotland to calculate the 
correct age allowance because of the extremely heterogeneous 
nature, in age, of the group of candidates, and the unduly wide 
age-range. But these difficulties may be diminished—they are 
much less in some counties—and are not insuperable. 

As long as examinations are held only annually, or in some cases 
every six months, it must be right to give an age allowance to make 
a child born in June have the same chance as one born in January, 
even if this apparently increases the number of misfits in the 
secondary school. Nor is the reason why it does so far to seek: 
it is probably because the secondary schools do not give an age 
allowance over the twelve months in judging a child. They 
agree with the entrance marks without age allowance because 
they themselves give no age allowance. Both procedures are 
unfair, and to that extent are in agreement. Age allowances are 
sometimes, by those opposed to them, called a premium on youth. 
They are not that. When scientifically applied they are a device 
to compensate for the unfair premium on age. 
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FINAL ORDER OF PREDICTIVE VALUE OF THE 
VARIOUS TESTS AND BATTERIES 

The Evidence 

To the original foUow-up correlations for senior and junior 
secondary schools we have now added the following evidence: 

Best weighted correlations. 

Correlations with the best battery. 

Number of misfits for each battery for the senior secondary 
group. 

For ease of comparison we transformed all the results into 
comparable correlations by methods already explained. 

TABLE XLUI 


Collected correlations \vith success for the various tests and batteries 


Battery 

Dun* 

tion 

10 

mmutei 

(0 

Senior secondary 
ichooh 

Junior secondary 
schools 

Misfits 

( 6 ) 

1 

Corre¬ 

lation 

with 

best 

battery 

(7) 

Average 

of (2). 

(4) and 
(6) 

(8) 

z 

( 9 ) 

Equal 

weighted 

(2) 

Best 

weighted 

(3) 

Equal 

weghted 

(4) 

Best 

weighted 

(5) 

IQ+Q+T* . 

♦ 9 

•804 

■SoS 

•806 

*808 

•802 

ft • 

•804 

1* 1 1 

IQ + Q+S + Ti 

♦ ♦ 

•800 

•80S 

•799 

«So9 

•802 

•799 

•800 

1*10 

Q +S +T$ . 


.790 

•794 

•784 

.787 

•793 

.787 

•789 

1*07 

IQ + Q . 

* 5 ° 

•786 

•791 

•774 

•777 

•792 

•784 

•784 

i-o6 

Q +T» 

% % 

.783 

•787 

•780 

•780 

•780 

•777 

•781 


IQ+Q+S . 

32s 

•783 

.796 

•773 

•780 

•779 

•782 

.778 

w 

1*04 

Q +S • . 

270 


•782 

•753 

.756 

•777 

•775 

•768 

1*02 

Q 

« 9 S 


# ft 

•738 

a a 

•760 

•762 

^756 

•99 

IQ+S * 

c 

»30 

•736 

•736 

.741 

•741 

•736 

.740 

•738 

•95 

0 * 

75 

*698 

ft ft 

.70s 

9 ft 

.701 

.715 

•701 

.87 


55 

•691 

ft ft 

.689 

ft ^ 

.690 

♦690 

•690 

.85 




FUtitioui batUfici 





IQ+T» 

^ 9 
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•779 

•795 

•797 

•787 

•794 

•787 

1-06 

XU + b + Ti , 


•781 

.785 

•793 

•802 

•782 

•787 

.7S6 

l•o6 

^ + 1 1. 

♦ • 

.764 

.765 

•775 

•777 

•770 

•768 

•770 

1*02 

At * 

» ft 

•720 

ft ft 

• 74 « 

ft % 

.763 

•724 

.741 

•95 
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We have labelled certain batteries ‘fictitious* for the following 
reason. Teachers* estimates should not be used unless they are 
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Fio. 42.-Gr.phlcal pr«™tation of oompkt. <iou a. to order of merit of the 

vdrioui tt%t% zud batteries. 

oroperly scaled on a uniform examination. They therefore imply 
thafthe results of a uniform examination are avatlable, and would 
u We need, in fact, retain no battery which contams Ts 

unless it also contains Q. 
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In view of the results given in Chapter X it is unlikely that best 
weighting wiU be used in practice; and correlation with the best 
battery cannot be regarded as an authoritative test. We have 
therefore treated these merely as supplementary evidence, and 
used the averages of columns (2), (4) and (6) to give the final order 

of predictive value in column (8). 

The final judgment is best made from the graphs in fig. 42, 
in which the determining measures are shown at the left and the 
supplementary ones at the right. 


Conclusion 

Fig. 42, the product of an immense amount of labour, is 
worthy of careful study. On all the determining tests IQ +Q + Ts 
comes out as the best battery, though its superiority over IQ + Q 
+ S +Ts is slight. The difference may, indeed, be due to the fact 
that, in the complete battery, scholastic attainment is over¬ 
weighted in relation to IQ, an idea which receives support from 
the best weighted correlations. 

After a distinct gap there is a set of four batteries between which 
the intervals are small. Yet the order is not in doubt. On all 
the determining criteria Q + S + Ts is the best of the four. IQ + Q, 
while somewhat lower in the correlation for junior secondary 
schools, is decidedly better than Q+Ts, which in turn is better 
than IQ + Q + S. 

Thereafter the order is perfectly clear. 

Turning now to the supplementary data on the right-hand 
side we find nothing to cast doubt upon our final order, and 
much to confirm it. The only discrepancy is the rise in standing 
of IQ + Q + S, due, in all probability, to the fact that scholastic 
attainment is over-weighted. 

The order and spacing in fig. 42 conform closely to those 
in column (8) of Table XLIII, and we may take the latter as 
the basis for the final graphical presentation of the order in 

fig- 43 - 

As far as the order of merit of the single measures is concerned, 
our finding is that a carefully set and corrected examination is the 
best basis for the prediction of a pupiPs success in the secondary 
school. Thereafter the order is, teachers* scaled marks, scholastic 
tests and intelligence quotient. 

We have now attained one of our main objects, which was to 
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determine the order of predictive value of the various combina¬ 
tions of estimates, examinations and tests as actually used. This 


CORRELATION 

BATTERY 

CORRELATION 

BATTERY 

•804 • 
•800 - 

• 10+ Q + TS 

• IQ+O+S-t-TS 



•789 • 

•7&4 - 
•781 - 
•778 • 

OVS+TS 

• O-vTS 

• 1^0.+S 

_tl_ 

. ia.+ Ts 

’ lO+S-frTS 

-768 - 

• 0+s 

770 - 

• S-»TS 

•756 - 

■ 

1 


738 - 

• 1 

1 

s 

• TS 

•701 - 

- s 



-690- 

• IG. 




Fig. 43.—Final order of predictive value of the varioui teiU and batteries. 


is the result which is of import to educational administrators; 
but, from other points of view, it is of interest to note the varying 
durations of the measures which, so far as they can be estimated, 
are given in the first column of Table XLIII. 


Significance of the Differences in the Final Order 

We have now to consider how far the differences in fig. 43 
can be regarded as real, and not due merely to the accidents of 
sampling. This is no easy problem, but we give the followng 
simple rule for finding whether the difference between the final 
correlations for two batteries is statistically sigmficant. 
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If the difference between the z values ' for any two batteries is 
greater than '04, the difference between the corresponding correlations 
is statistically significant.^ 

The fact that not all the gaps are statistically significant is not 
surprising, and is no reflection upon the precision of our results. 
The more we increase the number of steps within a restricted 
range the more difficult it becomes to establish the significance 
of any one; and it must be remembered that while statistical 
significance establishes a real difference, its absence does not prove 
that there is no difference. 

A large proportion of the mediate differences are significant. 
If, for e.xample, an Education Committee is considering whether 
to use IQ + Q or IQ + S, it will find from the Table that the differ¬ 
ence of z values is -ii, which is nearly three times the minimum 
for statistical significance. It is therefore left in no doubt that 
IQ +Q is the better battery. 

We believe that the graphs in fig. 42 will carry greater con¬ 
viction to the general reader than these statistical considerations. 
His confidence in the soundness of our final order will be deter¬ 
mined by the fact that we have three different determinations, 
two absolutely independent and one partly independent, and that 
there are only two small deviations from the absolute conformity 
of all three orders. 

^ Given in column (9) of Table XLIII. 

• If we have estimates of tqp for two samples of and «, from equally 
correlated populations we can combine them by the z transformation. The 
combined result will have the same accuracy as a single correlation obtained 
from +rt| - 3 pairs. 

In our investigation we combined correlations such as rqp^ and rqpj, 
but these were determined on the same pupils. We cannot therefore maintain 
that the result is equivalent to a single correlation based on nj + », 4 - n, - 6 pairs. 

In calculating standard errors we counted these as different estimates of the 
correlation for the same sample. 

In this way we obtained the standard error of the correlations for the senior 
and junior secondary schools. The correlations corresponding to the misfits 
are not wholly independent of those for the senior secondary schools, for they 
are based on the same group of pupils. They were, however, obtained by a 
different method, and we counted them as having the same standard error as • 
the correlations for the senior secondary group. Knowing the three standard 
errors we then calculated the standard errors of the correlations in column (7). 

The use of z values has the great advantage that the standard error of z is 
not dependent on the value of the correlation for the total population. 
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Suitability of the Various Batteries for Consideration of 
Border-line Cases: Effect on Order of Predictive Value 

We have seen that by the aid of certain additional principles in 
border-line cases we can reduce the number of misfits for the 
complete battery by 6. Now, if we use IQ only we cannot adopt 
any of these additional principles; with a pair like IQ + Q, only a 
limited number. This consideration must therefore be taken into 
account in fixing the final order. 

For the senior secondary group, omitting all pupils with absences 
in any item, we found the reduction in the number of misfits for 
all the batteries when the additional principles were applied. In 
so doing we introduced certain special rules to meet different cases. 
For instance, with a battery like Q+Ts, Tes and Tas were never , 
counted as flops, but an examination mark was omitted if it fell 
below the corresponding teacher’s scaled estimate by more than 
•7 sigma. 

The reduction in the number of misfits by the use of additional 
principles in border-line cases is as follows: 


Number of tests in 
battery 


Average reduction in number 
of misfits 


4 

3 

2 

I 


40 

27 

•3 

0-0 


The reduction is greater with the batteries which include the 
larger number of tests, and a study of the complete data shows 
that the only effect on the order of predictive value would be to 
raise IQ + Q + S a little in the scale. Apart from that, the result 
would be much the same as that obtained by stretching the line in 
fig. 43, keeping the lower end fixed. 

The detailed figures also suggest that additional principles 
should be used only with batteries of more than two tests. 


Recommendations 

We recommend: 

That all pupils at the Qualifying stage be tested, not merely 
the applicants for bursaries or for admission to senior 
secondary schools. 
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That the battery used in urban areas be I +Q+Ts. 

That in rural and other areas where satisfactory scaling ot 
teachers’ marks is impossible, the battery used be I +U. 

Possibility of Escape from External Examinations 

Our problem was to find the batteries of existing measures 
which are most effective in selecting pupils for secondary educa¬ 
tion. As scientists we should have had no feelings in regard to the 
answer; but, as educationists, alive to the evils of external examina¬ 
tions, our satisfaction at the consistency with which our experi¬ 
ments revealed the best batteries was not unmixed with regret 
that these all contained the Qualifying examination. Having 
stated our conclusions in a dispassionate scientific way, we may 
now give rein to our educational prejudices and consider whether 
there is any way of escape. 

Perhaps the most promising suggestion is that we should use 
an intelligence test plus teachers* estimates scaled on its results. 
We thus almost completely avoid the evils of external examina¬ 
tions, but this advantage would be purchased at a cost. 

In Chapter V we have given our reasons for believing that 
scaling upon intelligence score is unsound, of which the most 
important are: 

With this method of scaling, the marks for all pupils from a 
particular school might be in error by as much as 12 per 
cent. 

Since we should scale English and Arithmetic separately 
they would both have to be scaled upon intelligence score, 
and we know that the correlation between school means 
in English and Arithmetic is too low to justify this 
procedure. 

The correlation with success of the best battery is *804, and 
from fig. 43 it is clear that the correlation for the one now 
suggested will be less than +787. Keeping in mind all the facts 
already given on this point, we estimate that the number of 
misfits in our secondary group of 416 (56 for the best battery) 
would be increased by about 5. 

That is the debit side of the account; but against it we must set 
the educational advantages to the whole 3,000 pupils in the Qualify¬ 
ing group resulting from the removal of the incubus of external 
examinations from the primary schools. 


6 * 
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Science is not yet capable of putting the credit side into 
statistical terms, and the decision must be left to the wisdom of 
the philosopher. But the problem is well worthy of further 
investigation, particularly in the direction of devising means of 
reducing the debit. This might be effected by such methods as 
the following: 

Improvement of teachers’ standards of marking, for example, 
by encouraging them to make more use of standardised tests. 

Replacement of the present intelligence tests by some kind of 
composite test, which, while including English and Arith¬ 
metic, would be free from the evils of external examination. 

Increase in the flexibility of the secondary system in the first 
year in such a way that transferences could be easily made and 
mistakes in selection rectified. 



CHAPTER XVII 


PROBATIONARY PERIOD IN THE SECONDARY 

SCHOOL 

The following results have some relevance to the last suggestion 
made in the preceding chapter. 


Predictive Value of Marks in First Year of Senior 

Secondary Course 

Our method is to compare the prediction of the pupil’s later 
success made at the Qualifying stage with that which we could 
make during and at the end of the first secondary year. The 
correlations between marks in the first secondary year and later 
success for the senior secondary group are as follows: 

. , , Correlation with average class marks 

Average marks m 


1st term of ist year 
2nd term of 1st year 
1st year 


+ •854 
+ •886 
+ •894 


These data show the extent to which our predictions would 
improve throughout the first secondary year, and give us a 
statistical basis for considering the best length of the probationary 
period. 


Comparison with Prediction made at Qualifying Stage 

For this purpose we take the results for the best battery, 
IQ + Q+Ts, and we must use the second-year average marks, 
for which, as we have seen, the correlations were rather low. 
This ensures that the ranges are the same for the correlations 
which we are comparing. Making also the allowance for the 
reductions in number of misfits by the use of additional 
principles in border-line cases (translated into correlation terms 
by the methods of Chapter XIV), we find that the comparable 
correlation is 

'’aQ+Q+w,FA*“ +’829 
*7* 
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The position is therefore that with a good battery at the Qualify- 
ing stage we can reach a correlation of -829; at the end of the first 
term of the first secondary year the figure is only *025 higher. 
The difference is not so decisive as might have been expected 
when we consider how heavily the dice are loaded in favour of 
the later prediction. The decision is reached six months later, 
by which time the pupil has reacted to the new environment of 
the secondary school; it is based on a much larger number of 
tests, including those in the new subjects of the secondary curri¬ 
culum. These are legitimate advantages of the .prediction made 
at the end of the first secondary term, but there is a possible 
illegitimate one which arises from the halo effect. 


Maximum Correlation with Success to be expected at 

THE Qualifying Stage 

One very interesting deduction which we can make from the 
above results is that no improvement of testing at the Qualifying 
stage is ever likely to produce a battery giving a higher correlation 
than -854; for we could hardly expect that such a battery could 
yield a better result than a full set of examinations in the actual 
subjects and in the environment of the secondary school. 

We feel, indeed, that we can, without incurring a charge of 
undue complacency, hold that our correlation for IQ + Q+Ts 
must be very near to the high-water mark of prediction at the end 
of the primary school course. 

Minimum Number of Misfits to be expected with any 
System of Selection at the Qualifying Stage 

By the methods described in Chapter XIV we find that if the 
number of misfits for our best battery were 53, then: 

If the prediction were made on the basis of the first-term marks 
in the secondary course, this would be decreased by 4. 

If made at the end of the second term the decrease would be 8. 

If made at the end of the first year the decrease would be 9. 

We can also prepare the following Table showing what the 
complete results of selection would have been if made at the end 
of the first term in the senior secondary school. It is apphcable 
to conditions, likely to obtain throughout Scotland, where the 
distribution of applicants is skew and the proportion of unht 
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candidates is 25 to 30 per cent. We have therefore given the 
figures in percentage form: 



Successes 

Failures 

Total 

Admit 

. 669 

69 

73-8 

Reject 

3-9 

22-3 

262 

Total 

00 

0 

• 

292 

1000 


These figures are very important, in that they represent the 
best result that could possibly be obtained by any system of 
selection at the Qualifying stage; and they point to the following 
practical conclusions: 

It is most unlikely that, under the conditions stated above, any 
system of selection at the Qualifying stage will ever yield 
results where: 

(1) The number of misfits will be less than io*8 per cent. 

of the applicants for admission to the senior 

secondary courses. 

(2) Less than 9*3 per cent, of the number admitted will 

fail to make good.^ 

(3) Less than 3*9 per cent, of the applicants will be 

erroneously refused admission. 

Application of Foregoing Results to the Problem of 

THE Probationary Period 

Let us consider first the suggestion that the probationary 
period should be one year. We should then have 9 fewer misfits 
(44 instead of 53) when the final selection of course was made— 
a very considerable saving. But, as usual, there is a debit side to 
the account. If, in the probationary year, we do not introduce 
the new characteristic subjects of the senior secondary course, 
we have no real test of the child’s reactions to these subjects, and 
the prognostic value of the probationary year would be con¬ 
siderably reduced. If we do introduce them, we have to set 
against the saving of 9 misfits the serious wastage resulting from 
teaching these new subjects to a whole group, many of whom 
will give them up at the end of one year. 

An exact assessment of this wastage is beyond our scientific 
power, but we would emphasise one point and make one suggestion. 

^ Assuming, of course, that the correct pass-mark has been used. 
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Whatever procedure be adopted in regard to a probationary period, 
we cannot escape the necessity for making as careful a prognosis as 
possible at the Qualifying stage; for we must then decide which of 
our Qualifying group are to get the chance of taking it. To give 
the chance to all would be sheer educational folly. If a careful 
selection for the probationary year is made, we suggest that a 
compromise procedure should be adopted in regard to the content 
of the course, giving only one foreign language. Thereafter, all 
that we can do is to try to ensure that change of course, or even 
of type of school, is as easy as possible. 

Probationary periods of less than,a year have been tried. On 
this point our results indicate that the superiority of prediction 
after one term over prediction at the Qualifying stage is hardly 
sufficient to warrant the dislocation of organisation which such an 
arrangement would cause. A period of two terms, while giving 
a better basis of prediction, is unlikely to be considered. 



CHAPTER XVIII 
THE MISFITS 

After trying all possible methods, and using all the supplementary 
data at our disposal in regard to health, industry and personal 
qualities, we were still left with 51 utterly refractory misfits.^ 
There were 32 pupils who were failures in the senior secondary 
course, and, no matter how we used our data, they were admits; 
and there were 19 pupils who were rejects at the Qualifying stage, 
but who turned out to be successes. The time has now come to 
examine these with a view to finding pointers to the next steps 
in the improvement of our selection procedure. 

The Admit-Fails 

EXTENT TO WHICH THESE WERE BORDER-LINE CASES 

One would expect that the majority of these final obstinate 
misfits would be pupils close to one or both of the admit-reject 
and success-fail border-lines; but an examination of their Qualify¬ 
ing scores shows that this is not so: 


Qualifying score 
(IQ + Q+S+Ts) 

Boys 

Girls 

Total 

Number who were 
doubtful failures 

(sigma) 

+ 1-40-1-49 

« * 

I 

I 

in secondary course 
I 

+ 1-30-1-39 

2 


2 

» • 

+ 1-20-1-29 

4 

3 

7 

4 

+ 1-10-1-19 

2 

• % 

2 

• • 

+ 1-00-1-09 

4 

I 

5 

2 

+ 90- -99 

3 

3 

6 

3 

+ -80— -89 

4 

2 

6 

• # 

+ -70- 79 

2 

1 

3 

i e 

Total 

21 

11 

3 ^ 

10 


There is certainly no evidence of bunching at the admit-reject 
line; and if we regard as border-line cases those falling between 
+ •6 sigma and +-8 sigma, only three of the pupils would be 
counted border-liners on the entrance test. The last column 
^ This was the number for whom we had absolutely complete data* 

*75 
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indicates the number of cases, lo in all, where the head teacher 
gave his judgment as to success in the secondary course with 
hesitation: all the others were perfectly clear failures. We may 

therefore take it that only 13 out of the 32 admit-fails fall into the 
category of border-line cases. 


ILLUSTRATIVE CASES 

The girl whose profile is given in fig, 44 is a perfectly clear admit 
on all tests; the home conditions are quite good, and neither the 



Fic. 44.—The admit'fail with the highest Qualifying score: Failure due to health. 

primary nor the secondary head master has any fault to find with 
her attitude or interest. She herself was not particularly keen to 
take a senior secondary course; but she had no strong objection 
to meeting her parents’ wishes. 

We diagnosed this to be a case where the girl has been carrying 
on a losing struggle against indifferent health. At the Qualifying 
stage the Medical Officer’s report recorded anemia, defects of 
vision, and a general health grading of D. The secondary head 
master ascribes her failure largely to health reasons. 

Yet, while we are fairly certain that taking account of health 
would, with this pupil, have saved us from error, we have seen that 
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as a general principle such a procedure would increase the total 
number of mishts. The only suggestion for improved selection 
arising from our consideration of this case is that where there is 
ground for serious doubt as to health the pupil concerned should 
be specially examined by the School Medical Officer. 

We cannot see that we could have been blamed for admitting 
the boy whose profile is shown in fig. 45- There is not a single 
unfavourable indication in all his Qualifying data. His entry to 
the secondary school course seems to have been determined by the 



fact that he was advised by his primary head teacher to sit 
the bursary examination. He was successful; and one gets the 
impression that it was a Scottish reluctance to give up the bursary 
rather than any keenness for secondary education that decided 
him to avail himself of it. 

The view of the secondary head teacher was that the difficulty 
lay in the home. The boy got no encouragement from his parents, 
who seem to have soon regretted their decision to enrol him for 
the five-year course. As a consequence he had little interest in 
his studies and a strong desire to get away to work. One can well 
understand that this lack of home encouragement, while not 
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affecting the primary record, might have a more serious influence 
^vhen the boy approached the leaving age. 

In the puzzling case shown in fig. 46 something must have 
happened in the girl’s history at the beginning of the second year 
in the secondary school. She is a perfectly clear admit on the test 
results, is above average in practical ability and in health, is graded 
A in industry; and the primary teacher thinks that she will be not 
merely a clear but a distinguished success in the senior secondary 



school. The primary report mentions perseverance and self- 
reliance as her marked personal qualities. The home conditions 
were favourable on the whole, but she was ‘allowed to take life 
too easily.’ 

How then can we explain this remarkable drop to the level of a 
sure failure at the end of her first year? The only answer we can 
give is speculative. In discussing the pupil with the head master 
we were told that the drop was accompanied by signs of a definite 
change in personality, the details of which might give a clue to 
identification. The child was also handicapped by a certain 
deformity, perhaps not of much import at the primary stage, the 
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dawning sensitiveness to which might have serious effects on the 
temperament and character of an adolescent girl. 

Perhaps the main pointer here is that such pupils should be 
referred to a Child Guidance Clinic. At any rate, no improve¬ 
ment in tests and examinations will save us from wrong placements 
in cases of this kind. 


Fig. 47 gives the profile of a boy who is just above the admission 
mark but is a very clear failure in the secondary school. Two facts 



seem to have contributed to his lack of success. In the first place, 
he was ‘rather indifferent to school work* in the primary classes, 
his interests being mainly in games. He was not consulted in 
regard to his enrolment in a senior secondary course, went there 
with a grudge, objected to ‘too much home work,* and left to 
become a salesman. The second cause was no doubt his weakness 
in Mathematics. Asked in Miss Rodgers’s questionnaire ^ as to 
the subjects which he found difficult, he mentioned Mathematics, 
adding, ‘It all depends on the teacher . . .* He must have had 
something else of interest to add here, for he had started another 


* See p. 198. 
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clause, but unfortunately thought better of it. His profile shows 
that he is very good in English, but he just escapes the principle 
of the minimum of o sigma in Arithmetic. The clever boy can 
overcome such a weakness, but the mediocre or dull cannot; and 
continued failure and discouragement in one subject soon affect 
all the others. 

The Reject-Successes 

EXTENT TO WHICH THESE WERE BORDER-LINE CASES 


The Qualifying scores of the reject-successes were as follows: 


Qualifying score 
(IQ + Q + S+Ts) 

Boys 

Girls 

Total 

Number who were 
doubtful successes 

(sigma) 

+ -90—99 

• • 

1 

I 

in secondary course 

I 

+ •80—89 

• • 


• • 

• « 

+ 70-79 

I 


I 

• • 

+ -60—69 

2 

I 

3 

• • 

+ .50-59 

2 

2 

4 

I 

+ -40-49 

3 

I 

4 

3 

+ .30-39 

I 

I 

2 

I 

+ •20—29 

2 

1 

3 

2 

+ •10—19 

I 

• • 

I 

I 

Total 

12 

7 

*9 

9 

% 

^ ^ m 


The two pupils above + *70 sigma were rejects because they 
fell below the minimum of o sigma in IQ. The total number of 
border-line cases is 13. 


ILLUSTRATIVE CASES 

We now know that the pupil whose profile is shown in fig. 48 is 
a boy of intelligence and character, but we can see no way in which 
we could have admitted him on his Qualifying data. The primary 
teacher indicated self-reliance as a marked personal quality, and 
after reading his replies to Miss Rodgerses questionnaire we are 
inclined to agree. He did not sit the bursary examination, no 
doubt because financial assistance was not necessaiy. He con¬ 
curred in his parents’ view that he should take a senior secondary 
course, though one doubts whether the heads of the jumor 
secondary schools would accept the implications of his wording. 
*I entered a senior secondary course,’ he says, ‘because it is use¬ 
ful to have a good education in these days when one must be 
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Fio. 48.—A border-line reject but a clear »ucce$J. 



Fio. 49.—A border-line reject not much above the luccess line. 
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well-educated to find decent work.’ Choosing the French-German 
course in preference to French-Latin because he thought ‘a living 
language would be more useful in the present day,’ he has found 
no reason to regret his choice. The main reason for his success in 
the opinion of the secondary head master lay in the home; the 
parents gave him every encouragement and were keenly interested 
in his progress. We are glad that our rejection of this boy was 
merely fictitious, and that he has had, and taken, his chance. 

Fig. 49 illustrates the type of case which we should have expected 
to be much more common among those irreducible misfits. The 
boy is not far below the line for admission; indeed, if he had been 
just a little better in English he would have earned the lower pass- 
mark for special ability in one subject. He is also very little above 
the success line, and the head teacher gave his decision with some 
hesitation. 

The boy’s ambition is to become a doctor, but he is quite 
emphatic on the point that he is not in the right course. ‘Two 
languages,’ he says, ‘are too much for me.’ We hope he will win 
his fight which, but for good home conditions and encouragement, 
he would probably have lost. 

Fig. 50 gives the profile of a bespectacled little boy from a good 
home who did not sit the bursary examination. In the primary 
school his interest in school subjects, with perhaps the exception 
of Music, was about average, and his attainments were well below 
the admission line. The choice of a senior secondary school was 
his own, though one gathers that he did take some account of his 
parents’ wishes; and he is quite satisfied that it was a wise one. 

At first we suspected that the remarkable improvement at the 
end of the first year might be due to late development, but the 
head teacher was of opinion that the change was on the side^ of 
personality rather than of intellect. Just about the beginning 
of the second year the boy seemed suddenly to acquire a sense of 
responsibility and then began to apply himself to his studies as he 
had never done before. 

The type of case shown in fig. 5^ constitutes a difficulty in 
follow-up work. The boy is a clear reject, and we have, from the 
prophetic point of view, no cause to complain of his first yeairs 
performance. On repeating the first year he jumped up to a clear 
success level, and maintained his position reasonably well in the 
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Fic. 50.—A possible Mate developer.* 



Fig. 51.—A pupil who repeated the fint-jear coune. 


3 
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following year. Taking account of all the facts, the head teacher 
was of opinion that he would stand the pace throughout the course 
and obtain the Leaving Certificate within the six years. He is a 
steady persevering boy who comes from a good and interested 
home. 

Analysis of Causes of Discrepancy 

PROPORTION OF CASES WHERE DISCREPANCY MAY BE DUE TO 
UNRELIABILITY OF ADMISSION AND SUCCESS MEASURES 

Misfits may be due to lack of reliability of the measures or 
pass-marks for admission and success, or to extraneous factors like 
failure of health, deterioration of home conditions, etc. It is im¬ 
portant to find out the relative proportions of these, for this will 
be our main pointer as to directions of further advance. If the 
former number is large we must strive to improve our tests; if 
the latter, we shall have fuller knowledge of the limits of any 
predictive procedure. 


Possible cause of discrepancy 
Unreliability of measures 

Admit-fails 

Reject-successes 

Total 

Unreliability of entrance tests 

10 

4 

*4 

Unreliability of success grading 

3 

9 

12 


13 

13 

26 

Extraneous factors (such as health) 

*3 

5 

18 

No satisfactory explanation 

6 

I 

7 

Total 

11 

22 



The allocations were made with caution. To give an idea of 
the standards used, we may mention that the pupil of fig. 44 
was classed as one where the success grading might be wrong; the 
pupil of fig. 47 as one where the entrance mark might be at 
fault. Further, unless we were absolutely satisfied that we had 
a convincing explanation on the ground of extraneous factors we 
relegated the case to the category, ‘no satisfactory explanation. 

Of the 51 misfits 26 might be due to fallibility of the entrance 
or success measures: in 7 cases we have no satisfactory reason. 
These are almost certainly over-estimates; so we have full con¬ 
fidence in concluding that in at least one-third of the cases the 
discrepancy is due to extraneous factors and could not be avoided 
by any improvement of the tests and examinations. 
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EXTRANEOUS FACTORS CAUSING DISCREPANCIES 

« 


We selected in each of the 18 cases what we thought to be the 
main extraneous factor causing discrepancy and found the position 


to be as follows: 

Cause of 
discrepancy 

Home conditions 
Desire to take up 
work 
Health 

Unsuitable course 
Personal qualities 
Changes of school 

Total . 


Admit-fails 

Boys fjirls Total 
3 > 4 

2 I 3 

2 .. 2 

I 1 2 

I .. I 

I .. I 

10 3 »3 


Reject-successes 

Total 

Boys Girls Total 

123 7 



I 

I I 

I 


3 

3 

2 

2 

I 


2 


3 


5 



In any one case, however, there are usually several contributory 
causes, and we present below the number of times that the various 
factors were mentioned in the reports. This gives a fairer measure 
of their relative importance: 



Number of times mentioned 


Factor 

Admit-fails 

Reject-successes 

Total 

Home conditions 

12 

10 

22 

Unsuitable course 

10 

• • 

10 

Health 

6 

2 

8 

Personal qualities 

6 

1 

7 

Desire to take up work 
Interest in a particular 

4 


4 

subject . 

• « 

4 

4 

Changes of school 

2 

«• 

2 

Total . 

40 

»7 

57 


It will, of course, be understood that by health and personal 
qualities we refer to changes which took place after the Qualifying 
stage. The category, ‘interest in a particular subject,* may be 
illustrated by a girl of indifferent Qualifying attainments who was 
talented in Art and desired to become an Art teacher. She realised 
that a successful secondary course was a necessary part of the road 
to her goal, and this provided an incentive which pulled up her 
work in all subjects. 
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Suggestions for Improvements in Methods of Selection 

The smallness of our numbers, added to a consciousness of the 

deficiencies of our technique, precludes any definite conclusions. 

Yet our experience in working with these cases for nearly six years 

suggests the following directions of improvement in methods of 
selection. 

Reports from the Primary Schools 

We assume that the primary teachers will be encouraged to 
shoulder more and more of the responsibility for selection and 
guidance; and that, by a better technique than ours, they will be 
invited to give reports on home conditions, health, interests and 
personal qualities. In our view it would be both unnecessary and 
undesirable to ask them to make such reports on every applicant 
for admission to a senior secondary course. It is much better to 
have full and careful reports on a few selected cases than general 
reports on all. The knowledge that these reports would be 
carefully considered would increase their reliability. 

Special Medical Examination in Doubtful Cases 

If the primary teacher expressed doubts as to the child’s physical 
fitness to face the strain of a senior secondary course, the pupil 
should be referred for special examination to the School Medical 
Officer. 

Enlistment of the Resources of the Child Guidance Clinic 

If the primary teacher was of opinion that the child’s success 
might be prejudiced by the home conditions, these could be 
reported on by the Social Worker of the Clinic. Cases of difficulty 
in regard to personal qualities or cases of doubtful IQ could with 
advantage be referred to a Clinic Psychologist, who should also 
be consulted when a child suddenly changes in personality or in 
class standing during the secondary course. 

Fuller Use of the Flexibility of the Regulations for Senior Secondary 
Courses 

The number of misfits arising through unsuitability of the 
course will be reduced if the Education Committee uses a good 
battery of Qualifying tests and sees that the results are sent to the 
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secondary head teachers: but certain changes within the secondary 
school are also needed. In particular, there might be a reduction 
in the number of pupils who are encouraged to take two foreign 
languages, a greater provision of courses for adolescents whose 
cultural tastes are not met by the old ‘normal general course, and 
a greater effort to exploit the incentive that comes from special 
talent in subjects like Art and Music. 

Extraneous Factors affect Boys more than Girls 

One curious feature of our results is that in the misfits the boys 
always outnumber the girls, particularly in the admit-fail category. 
Here the ratio of boys to girls is 21 : 11 for the Senior Leaving 
Certificate, and 19 : 8 for the Junior Leaving Certificate. This 
seems to indicate that a girl tends to work up to her capacity 
better than a boy. It is possible, too, that a girl is not so 
liable to the discontent and unrest that come over so many boys 
in adolescence and lead to a desire to get away from school at 
all costs. 



CHAPTER XIX 

« 

BURSARY EXAMINATIONS 


The object of bursaries for senior secondary courses is to ensure 
that no child who shows promise of profiting by such an education 
will be debarred therefrom by reason of the cost; and in the City 
where the Inquiry was conducted awards were, in 1935—1936, 
made on the basis of a special bursary examination. The numbers 
of candidates and of bursaries awarded were: 


Group 

Candidates 

Bursaries awarded 

December 1935 

140 

S 3 

June 1936 

275 

130 


Is A Special Bursary Examination necessary? 


It has been a fairly common practice in Scotland to set a special 
examination on a higher standard for the award of bursaries. 
This was no doubt based partly on the belief that a stiffer examina¬ 
tion would give a wider scatter of the good pupils, and ensure that, 
at the pass-mark, fewer candidates would have the same score. 
Clearly, if the examination were so easy that the clever pupils 
all scored full marks, this extreme ceiling effect would make the 
results useless for bursary purposes. • 

# 

THE CEILING EFFECT 

So far as we can see, the fear of a ceiling effect is no sound reason 
for a special bursary examination. In the first place, it is quite 
possible to have an examination for a complete Qualifying group 
which has neither a ceiling nor a floor effect. Our own examina¬ 
tion showed neither; and reference to fig. 56^ will show that 
this holds also for the distribution of scores on the complete 
battery. In the second place, the ceiling effect would be no dis¬ 
advantage if the pass-mark fell some way below the point where 
the frequency was greatest. 

1 P. 197. 
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SPREAD OF CANDIDATES AT THE PASS-MARK 


This consideration is certainly important; but it is rash to 
assume that a stiffer examination will necessarily effect an im¬ 
provement; it may have the opposite effect. 

While the fact that the bursary histogram in fig. 52 is lower on 
the scale is not conclusive proof that the e.xamination was stiffer, a 
study of the papers confirms the belief. It will be noted that the 
frequencies at the verticals which cut off the actual number of 
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Fio. 52 .—Score-»catter of December bumry candldatei on the burwry examinatioa 

and on the Qualifying ezanunatioo. 


^ccessful candidates are very nearly the same in both examinations. 

The stiffer examination has therefore failed to secure a better spread 
at the critical point. ^ 

Fig. 52 shows how much of the effectiveness of the stiffer 
examination in spreading the candidates at the pass-mark defiends 
upon the point at which the pass-mark is drawn. If it had been 
necessary to select less than 6 candidates, instead of 53, the stiffer 
examination would have been the better; but for almost any 
other number the easier examination would have been as good a 
basis of selection. Now, in Scotland it is never likely that only 
6 candidates out of 140 would be entitled to bursaries; and we 
may therefore conclude that no statistical advantage is gained by 
a social examination of the standard used in December lotc 
It ... of course, possible that a stiffer examination iTgJv^’a 
better order of merit, though we think that tbi. i. 






190 SELECTION FOR SECONDARY EDUCATION 


Arguments against the Special Bursary Examination 

Our view is that a special examination is neither necessary nor 
desiMble; and that bursaries should be awarded on the basis of 
the Qualifying scores obtained by the appHcation of a good battery 
to the whole Qualifying group. In arriving at this conclusion we 
had the following considerations in mind: 

Qualifying scores for the complete group are required, not only 
for the award of bursaries, but also 

(a) For guidance and selection for post-primary courses. 

{b) As a basis for classifying entrants in the senior and 
junior secondary schools. 

If teachers’ estimates are to be used, as is very desirable, they 
must be scaled; and the scaling is possible only on the basis 
of an examination of the complete group. 

Without Qualifying scores for the complete group there is no 
satisfactory means of fixing the pass-mark either for bursaries 
or for entrance to senior secondary courses. 

A single examination is unjustifiable as a basis for the award 
of bursaries for secondary education. 

Such an examination has no advantage over the Qualifying scores 
either in absence of ceiling or floor effects, or in spread of 
candidates at the pass-mark. 

The institution of a special bursary examination offends the 
principle of ‘parsimony of examinations,’ which should have 
the same honour in the eyes of the educationist as ‘parsimony 
of hypotheses’ in the eyes of the scientist. 

Correlations with Bursary Examination 

Dr McIntosh calculated the correlations of the bursary examina¬ 
tion marks with IQ and Qualifying score for the December and 
June groups, and found them to be as follows: 

Correlation between bursary December June 

examination and (N = i4o) (N = 275) 

IQ +-501 +595 

Q +-591 +'544 

The fact that the correlations with IQ are not so high as the 
corresponding figures for our own examination is partly due to the 
smaller standard deviation of the bursary group. 

The most interesting point is that the correlations between 
the bursary examination and our own are as low as ‘591 and * 544 * 
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AN APPLICATION OF THE GENERAL PRINCIPLES OF CHAPTER XIV 

With a correlation of '591 between the two examinations the 
discrepancies would be very- great, as the reader will by now realise. 
If, however, he has read Chapter XIV he should also be able to 
translate this indefinite statement into a fairly exact one; and 
by so doing he will have an opportunity of testing the usefulness 
of the methods therein described. 

The following is a summary of the steps. We are selecting 
53 candidates out of 140, so the number of discrepancies for zero 
correlation would be From fig. 30 we see that the number 
for a correlation of *6 would be 62 per cent, of this, namely, 41. 
That is, 20 candidates who get bursaries on our examination 
would be refused awards on the bursary examination; while 20 
who were refused on our examination would receive awards on 
the bursary examination. The actual number was 19, showing 
that the method gives quite a good estimate. 


If an Education Committee has ascertained the Number of 
Successes among the Candidates awarded Bursaries it 
can estimate the Number of Potential Successes among 

THOSE WHO WERE REFUSED 


^ of this section had not been so long we should have 
added and it would be most salutary for it to do so.’ As a rule 
the candidates who are refused bursaries do not go into senior 
secondary courses. A large proportion of the mistakes made by 

^hools; they do not obtrude themselves on the notice of the 
Committee, and this blind spot in its administrative procedure 
y give rise to an attitude of unwarranted complacency It 
also gives the Committee Uttle opportunity of knowing wheth« 
the pass-mark is fixed too high or too low. 

The soundest method is, of course, a follow-up of the complete 
group of candidates; but this involves a considerable amouL of 

of Ednc:r wS 

DeTembrn^ ^-1“- U^.ng rn^ 

.0 faUntes and^^snSs " r^r^d?™Lm^X 
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resembles that used in the reconstruction of a prehistoric animal 
from a few bones. 

All we know is the pass-mark AB (fig. 53), the numbers 10 and 43, 
and the total number of candidates, namely, 140. Our aim is to find 
an estimate of the number which should go into the quadrant BED. 
We must, of course, assume normality of the distributions, and we 
must have an estimate of the correlation between the bursary 
examination and success. The latter need not be very exact for 



Fig. 53.—Ettimate of number of successes among candidates refused bursaries. 

this purpose, and the estimate could easily be made by any Director 
of Education who has been using scientific methods in his Area. 
In our own case tjie estimate would be + 7. 

Using the notation of Chapter XIV, we know that 

r = 7 

k= +* 3<7 

CEB =307 per cent. 

Now, knowing the volume CEB and the position of one of the 
planes which bounds it, namely, EB, it is possible by the method 
indicated in Chapter XIV to find the position of the other, namely, 
CE. We then find that h = oay from which it follows that the 
number which should go into the quadrant BED is 27* 
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The conclusion is that 27 out of the 87 pupils who were refused 
bursaries would have made good in the senior secondary school. 
The number of correct awards was 43, and the number of 
mistakes 37. It will now be understood why we recommend this 
as a salutary type of calculation for Education Committees; but 
to allay possible suspicion of the mathematical character of the 
method, we proceed to give the actual numbers. 

Analysis of Bursary Awards in the Light of later 

Success 

From our follow-up data we know the number of successes and 
failures, both among those who were awarded and those who were 
refused bursaries. 

TWO SAMPLE CASES 


We first give two sample cases in which we compare the predic¬ 
tions made on the basis of our complete battery and of the special 
bursary examination. Both are from the December group. 



Flo. 54.—A pupil who was awarded a bunar74 


The first, whose profile is shown in fig. 54, is that of a boy w-ho 

was thirty-ninth on the bursary list and therefore received an 
award. 


7 
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It will be noted that, on our Inquiry results, we should have 
had no hesitation in refusing a bursary to this pupil; and his 
performance in the senior secondary school shows that our decision 
would have been correct. 

For comparison we give, in fig. 55, the profile of a pupil who was 
sixty-ninth on the bursary list and was therefore refused an award. 



This boy, to whom we should willingly have granted a bursary, 
went on to a senior secondary school at his parents* expense. 


COMPLETE RESULTS 

The fitness of bursary candidates of both December and June 


- 0 

groups for senior secondary education is 

Successes 

shown below: 

Failures 

Total 

Awarded bursaries 

155 

28 

183 

Refused bursaries 

69 

163 

232 

Total 

224 

191 

415 


The result is enlightening. Of the 183 candidates awarded 
bursaries 28 were unfit for senior secondary education; 13 of these 
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28 were, in fact, judged to be unfit even for a three-year junior 
secondary course. Even more disturbing is the fact that no less 
than 69 pupils who were fit to profit by a senior secondary educa¬ 
tion may have been debarred therefrom by reason of the cost. 
The total number of mistakes is 97 in all. 

For the December group the actual number of refuse-successes 
is 26. The theoretical estimate in the last section (27) was there¬ 
fore a close one. 

We trust that the reader has by this time not merely a vague 
suspicion that the pass-mark in these bursary examinations was too 
high, but has proved that this must be so. Since the number of 
successes is greater than the number of failures, the number of 
award-fails should be greater than the number of refuse-successes 
at the correct pass-mark. 

The Pass-Mark in Bursary Examinations 

We shall assume that the battery I + Q + Ts is used and that the 
pass-mark should be determined solely on the basis of the fitness 
of the candidates to profit by a senior secondary education. 

We have shown that, on this battery, a candidate for admission 
to the senior secondary schools has an even chance of success if his 
Qualifying score is +*70 sigma. It might therefore be expected 
that the same pass-mark should be used for the award of bursaries; 
but this does not follow. The position of the correct pass-mark 
depends on the nature of the group, its social class, home condi¬ 
tions, and so on. We found, for example, that the point at which 
the child in the whole Qualifying group has an even (statistical) ^ 
chance of success in the senior secondary course is not + *70 sigma 
but +1*34 sigma. 

Now, for bursary candidates the extraneous conditions will be 
more favourable than in the complete Qualifying group, for in 
each case there is evidence of a desire for secondary education 
and the probability of home encouragement. Yet the home con¬ 
ditions and facilities for study will not be quite so favourable as 
in the group of candidates for admission to the senior secondary 
schools, many of whom come from homes which need no financial 
help. We should therefore expect that the pass-mark for the 
bursary candidates would lie somewhere between +*70 sigma and 
+ 1*34 sigma. 

We determined the correct pass-mark (that is, the point at which 

* See pp. 214-215. 
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half of the bursary candidates would be successes) by the methods 
already explained, and found it to be +*975 sigma. This means 
that the pass-mark used in the special bursary examinations was 
at least 6 per cent, too high. We explain in Chapter XXV how 
this pass-mark can be used in practice, but it should be remembered 
that it holds for the battery IQ + Q +Ts. With a poorer battery 
the pass-mark would be lower. 

Results if Bursaries had been awarded on Basis of Best 
Battery with its Correct Pass-Mark 

In awarding these bursaries we should have selected the battery 
IQ+Q+Ts; and, not having any financial responsibility, we 
should have used the correct pass-mark. Had we done so, the 
results would have been as follows: 


1 

Successes 

Failures 

Total 

Awarded bursaries 

192 

4 * 

233 

Refused bursaries 

32 

150 

182 

Total 

224 

191 

415 


This should be carefully compared with the actual awards given 
on p. 194. Short-sighted ratepayers might be inclined to cavil 
at the expense of the 50 additional bursaries that we should have 
awarded; but, taking a wider social view, we find some com¬ 
pensation in the facts that we should have admitted 37 more 
pupils who would have been successful in a senior secondary 
course, and that we should have had 24 fewer mistakes. 

We believe that these figures represent the best result that could 
be expected in the award of bursaries under present conditions. 



CHAPTER XX 

SURVEY OF THE QUALIFYING GROUP 


For our survey we use as a basis the average standard scores on 
the complete battery IQ + Q + S +Ts» which we call the Qualifying 
scores. 



C^UALIFYINO SCORE 
(sigma) 

Flo. 56.—Distribution of Qualifying scores for complete Qualifying group. 


Comparison of Boys and Girls 

COMPLETE QUALIFYING CROUP 

There is no significant difference between boys and girls, either 
in mean score or in scatter. The difference of means is *017 
sigma ^ in favour of the girls, and the standard deviations are *99 
sigma and *96 sigma for boys and girls respectively. 

^ This is about half the standard error of the difference. 

*97 
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SENIOR SECONDARY GROUP 

^ The mean score for the boys is +i-oi8 sigma and that for the 
girls + I-208 sigma. The difference is more than three times its 
standard error, and is therefore statistically significant. 

In an earlier investigation by Mr Robert Robertson it was found 
that the girls were better than the boys in inteUigence and 
scholastic attainment separately, and that this result obtained 
not merely for the group as a whole, but in all but one of the 
individual schools. The school which was the exception was the 
smallest. 

JUNIOR SECONDARY GROUP 

In the junior secondary schools the boys are slightly better than 
the girls, the means being +*167 sigma and +*125 sigma. The 
difference is, however, not statistically significant. 

If these tendencies should be confirmed in other years and 
Areas it would be of some interest to consider why the senior 
secondary courses should attract fewer of the clever boys than of 
the clever girls. Part of the answer may be that the roads to 
the vocations which attract the clever girl lie through the senior 
secondary school; but it may also be that the drags and pushes of 
the social environment, whose resultant is directed away from full 
secondary education, have more effect on boys than on girls. 

Factors which Determine Choice of Secondary Course 
(In collaboration with Miss Hilda C. Rodgers, M.A.) 

The exacting nature of the main investigation did not permit 
us to make a full study of this interesting topic, but Miss Rodgers 
obtained some preliminary results from a questionnaire given to 
391 senior secondary, and 420 junior secondary, pupils. Ad¬ 
mittedly the problem cannot be solved in this way; the children 
do not know all the facts; we get little light on the motives which 
influenced the parents, or on the extent to which poverty debars 
pupils from senior secondary education. 

choice of type of school: senior secondary or junior 

secondary 

One of Miss Rodgers’s questions to the senior secondary pupils 
was, ‘Why did you take a senior secondary rather than a junior 
secondary course?’ and she gave a choice of three answers: 

‘ {a) Because I was advised to go to a senior secondary school by my 
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primary head master, {h) Because my parents wished me to go to 
a senior secondary school, (c) Because I myself desired to go to a 
senior secondary school.’ She also left a space for Other reasons. 
We thought that this would seldom be used, but it elicited a large 
variety of frank responses, given with a sublime disregard for the 
requirements of e.xact statistics. For many pupils the choice was 
determined simply by the fact that they had been in the primary 
division of a senior secondary school. As one casual young lady 
put it, ‘I was in . . . and just carried on.* For others, the fact 
that a senior secondary' school happened to be the one nearest to 
the child’s home was the main consideration; quite a number of 
young educational oracles had the view that a senior secondary 
school gives a * better education’; a fair number admitted candidly 
that they went to a senior secondary school simply because their 
friends had gone there; one or two, apparently with snobbish 
propensities, were most influenced by the fact that they would 
mi.x with pupils of a better social class. 

Pupils were asked to indicate the main reason for the choice, 
and we give below what statistics we were able to extract: 


Reason for choice of t)'pe 

Senior secondary 

Junior secondary 

of school 

group 

group 

Advice from primary school 

28 

12 

Pupil’s desire 

70 

144 

Parents’ desire 

196 

95 

Other reasons 

78 

121 

Total 

.^72 

372 


Even if we allow for their unreliability, the figures suggest that 
only to a limited extent is advice from the primary school a deter¬ 
mining consideration in the child’s choice of type of secondary 
school. Parents’ ambitions and aspirations appear to be the main 
factor in the senior secondary group, and the pupils’ own tastes in 
the junior secondary group. Confirmation of this was given in the 
answers to another of Miss Rodgers’s questions: ‘ Were your wishes 
taken into account when your secondary course was decided 
upon?* To this question 94 per cent, of the junior secondary 
pupils gave an affirmative answer; in the senior secondary group 
only 73 per cent. 

Of the 121 in the junior secondary group who gave ‘other 
reasons,’ 59 indicated that they wanted to leave school at the 
earliest possible date, and 20 gave as their reason that they had not 
been awarded bursaries. The others defied classification. 
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CHOICE OF COURSE WITHIN SENIOR OR JUNIOR SECONDARY 

SCHOOL 


Miss Rodgers asked a similar question on this point which 
brought out the following replies: 


Reason for choice of 

Senior secondary 

Junior secondary 

course 

group 

group 

Advice from primary school 

15 

12 

Advice from secondary school . 

69 

7 

Pupil’s desire 

83 

149 

Parents’ desire 

104. 

98 

Other reasons 

89 

73 

Total 

A A A ^ 

360 

339 


Again we see that the pupils’ own desires have more influence 
in the junior secondary group and those of the parents in the 
senior secondary group. It is gratifying to find that advice from 
the secondary head master is an important factor in the latter. 

That old traditions die hard is shown by the fact that no less 
than 3 pupils chose their course because of the ‘value of Latin.* 


TRANSFERENCES BETWEEN SENIOR AND JUNIOR SECONDARY ' 

SCHOOLS 

Our results show that ease of transference from one type of 
school to another, when a mistaken choice has been made, is 
always likely to be an essential feature of an educational system. 
It is therefore interesting to find how far advantage is taken of 
such facilities for transfer as the present Scottish system offers.^ 

Miss Mitchell collected particulars of every change made by 
pupils in our Qualifying group and found the position to be as 
follows: 

Not a single senior secondary pupil changed to a junior secondary 
school. Apparently when these children find that they are in the 
wrong school they simply leave at the minimum leaving age. 

Only 12 pupils (6 boys and 6 girls) out of the 2,012 who went 
to junior secondary schools changed to a senior secondary school. 
All had spent a complete year in the first type, and all had to 
repeat the first year in the second type; so that for all these pupils 
there was a loss of a year, 

^ Circular No. 44 ^ Scottish Education Department. 
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The point is of sufficient interest to warrant the presentation of 
Miss Mitchell’s complete data: 


Pupil 

No. 


Quali¬ 

fying 

score 

(sigma) 


Course in 

Junior Senior 

secondary secondary 

school scliool 


Lengtli of 
course in 
senior 
secondary 
school 


Success or 
failure in 
senior 
secondary 
school 


1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 


+ 1-51 

Commercial 

French- 

German 

Still on roll 

Success 

+ 1-28 

Girls’ technical 

French 

Still on roll 

Success 

+ 123 

Commercial 

French 

1 4 years 

Success 

+ 1-20 

Commercial 

French 

Still on roll 

Success 

+ 119 

Commercial 

French 

I year 

Success 

+ 1-12 

Commercial 

French- 

Handwork 

1 year 

Success 

+ 1-00 

Commercial 

French- 

German 

Still on roll 

Success 

+ -94 

Commercial 

French 

Still on roll 

Success 

+ -79 

Commercial 

French- 

Handwork 

Still on roll 

Success 

+ *65 

Boys’ technical 

French 

Still on roll 

Failure 

- -05 

Boys’ technical 
Boys’ technical 

French 

Still on roll 

Failure 

- -02 

French 

I year 

Failure 


The Qualifying score is that on our complete battery, and it is 
comforting to note that with our pass-mark of + *70 our prediction 
would have been correct for every one of these pupils. We should 
certainly have discouraged the last two from making the change. 
Nine out of the 12 have made good in the senior secondary school, 
and 6 are likely to complete the course. In the light of these 
figures it would seem that the flexibility of our system is not so 
great as is sometimes thought. 


CHANCES OF COURSE WITHIN SENIOR AND JUNIOR 

SECONDARY SCHOOLS 

Miss Rodgers’s questionnaires were given i year and 8 months 
after the pupils had started the secondary courses, and certain of 
her questions bring out the flexibility of the system within the 
two types of school. 

She found that of the 391 pupils in her senior secondary group, 
58, or 14-8 per cent., had changed from one course to another, 
the main change being from a two-language to a one-language 
type. No less than 76 per cent, of the 58 changes were of this 
kind; 9 per cent, changed from one to two languages; and the 

7* 
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remainder were mainly due to the dropping of technical or 
domestic subjects. 

Another interesting point was that about one-fifth of the pupils 
‘were still dissatisfied with the course they were in. Most of these 
discontents were in the two-language courses and felt that they 
would have been happier and more successful if they had taken a 
technical or commercial course, usually the latter. Indeed, 15 
pupils, who were satisfied with their choice, added a note to the 
effect that their literary courses would have been improved by 
the addition of some commercial work. This adding of outspoken 
critical notes was a common feature of the questionnaires; and we 
read the pupil’s educational views with an interest not unmixed 
with admiration for their soundness. 

These results confirm the conclusions recorded later as to the 
number of pupils who should be encouraged to take two-language 
courses. They also suggest that the slow growth in popularity 
of the newer types of secondary course is not the fault of the 
pupils. Two further points that may be of interest to those who 
plan the courses are that a large number of senior secondary 
pupils taking a commercial course expressed the view that com¬ 
mercial subjects should be started in the first year, and that many 
pupils chose a junior secondary course rather than a senior 
secondary commercial course because, in the former, they could 
begin the commercial subjects earlier. 

Of the 420 pupils in the junior secondary group only 17, or 
4 per cent., had changed their course. Practically all possible 
transfers were represented, the commonest being from the com¬ 
mercial to the technical. Seventeen pupils were still dissatisfied 
with their course. 

Many of the facts elicited by Miss Rodprs’s questionnaires 
have a bearing upon the problem of the omnibus school. 

Comparison of the Four Groups at the Qualifying Stage 

QUALIFYING SCORES 

The four groups were senior secondary, junior secondary, back¬ 
ward and retained, and the distributions of the average scores on 

our complete battery are shown in fig. 57. _ . 

These figures show the way in which the nation s resources ot 
talent distribute themselves when almost free choice of school and 
course is left to parents and pupils. They are puzzhng as weU as 

illuminating. 
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The broken vertical is drawn at +70 sigma, the point above 
which the candidates for a senior secondary course have more 
than a 50 per cent, chance of success. VVe may therefore assume 
that all pupils above this line have the ability to profit by a senior 
secondary course. A glance at the histograms then shows that, 
while the 6 best pupils have all gone to the senior secondary school, 



OyALIPYlNG SCORE 
(sigma) 

Flo. 57.—DutributioQ of Qualifying scores for the senior secondary, junior secondary, 

back>vard and retained groups. 


a very large proportion of those who had the necessary ability have 
not taken advantage of this educational opportunity. 

The next striking point is the tail of the senior secondary group 
and the distance it stretches below the success border-line, 
penetrating to near the bottom of the junior secondary group and 
to about the mean of the back^vard group. 

The lower part of the distribution of the backward group is in 
conformity with expectation, but those in the top intervals would 
seem to have deserved a better fate. Why, one wonders, should 
6 pupils with attainments up to the average of the whole Qualifying 
group be sent to these special classes? For only one did we 
find an explanation: the girl simply refused to go to a junior 
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secondary school. Of the other 5, one was promoted to a junior 
secondary course, the others were retained in the backward classes, 
where the head teachers felt that their educational needs were 
satisfactorily met. 

The overlapping of the retained and secondary groups is almost 
incomprehensible. Three pupils who fall within the top 23 per 
cent, of the whole group are retained, while 3 who fall into the 
bottom 16 per cent, are sent to senior secondary courses. 

The means and standard deviations of the Qualifying scores of 
the four groups were as follows: 


Group 

N 

M 

a 

Senior secondary 

452 

+ I-I02 sigma 

•Sy sigma 

Junior secondary . i 

.975 

-f -146 

•65 

Retained 

278 

-1167 

-65 

Backward 

316 

-1-429 

*58 

INTELLIGENCE 


an early preliminary 

study 

made before the 

data for the 

^ A A 


retained group were available, Mr Robert Robertson found the 

Mean IQ 

1157 
102-0 
84-0 


following results for intelligence: 

Group 
Senior secondary 
Junior secondary 
Backward 


HEALTH, INDUSTRY AND INTERESTS 

Mr Robert Robertson found the percentages of the groups who 
were awarded the gradings A, B, C, D and E in each of these 
factors. To reduce the results to a simpler and clearer form we 
calculated average scores by giving five points for an A, four for a B, 
and so on: 


Group 

Health 

• 

Industry 

Senior secondary 

3-32 

3-66 

Junior secondary 

3-22 

3*22 

Backward. 

3-13 

2-30 


Interests 


Humanistic 

Realistic 

Practical 

363 

3.69 

3-48 

3-20 

324 

324 

215 

2 *i6 

2-47 


These results have not the reliability which would warrant an 
attempt to calculate the statistical significance of the differences 
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shown. Yet the figures are not without interest, whether we 
compare the rows or the columns. In interpreting them it is well 
to keep in mind that the score for a normal distribution would 
be 3-00. 

In health the senior secondary pupils are the highest, and the 
standard falls progressively from the junior secondary to the back¬ 
ward groups, a result which bears out the view that intellectual 
and physical fitness are associated. This is probably the most 
reliable of the results, for it was based on careful comparable 
gradings by the School Medical Officer. For industry, where the 
fall from group to group is more striking, we have to realise that it 
must be difficult for the teacher to grade without being influenced 
by the pupil’s school attainment. Also, the disentangling of cause 
and effect would be possible only with a much more careful 
technique than we were able to adopt. 

We have already given reasons for distrust of the reliability of 
the interest gradings, but from the first row we see that the pre¬ 
dominant preference of the senior secondary group is not for the 
humanistic but for the realistic. This may bring some encourage¬ 
ment to those who believe that the curricula of these schools are 
too literary and linguistic, and that realistic and practical subjects 
should have a more prominent place in some of them. These 
senior secondary pupils are, indeed, more interested in practical 
subjects than the pupils in the junior secondary group, whose 
tastes are more evenly distributed. On the whole, however, the 
interests of the latter are rather realistic and practical than 
humanistic. 

The interests of the backward group presumably lie elsewhere 
than in the world of the school, for they are well below 3-00 in all 
types of subject. Yet they make their highest score in the practical 
group, which supports the belief that the curriculum for pupils of 
this type should consist largely of manual activities. 


Comparison of the Sections taking different Courses 
IN THE Senior Secondary Schools 

The main interest of this comparison is that it allows us to judge 
how far the newer senior secondary courses had, by 1935, begun 
to attract the abler pupils. When the Latinless section of the 
French lycee was abolished in 1923, a professeur protested that this 
section had formed a convenient depository into which, with a 
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light heart, were dumped the pupils with insufficient ability to do 
well in classics: and he asked petulantly, ‘What is now to be done 
with these undesirables?* To find whether the modem and 
technical courses perform the same service in Scottish schools we 
prepared the following comparison: 


Section 

N 

Mean Qualifying 
score 

Mean IQ 

Latin-French . 

151 

+ 1-367 sigma 

1200 

French-German 

no 

+ 1-315 

1190 

French-Technical . 

74 

+ i-oo6 

114-7 

French-Literary 

100 

+ -619 

io8‘0 


In the French-Technical category we included French-Domestic 
and French-Commercial, and it is clear that such courses are 
beginning to attract a fair proportion of the clever pupils. The 
dumping-ground for weak entrants would appear to be the French- 
Literary course. 


Comparison of Sections taking Boys’ Technical, Girls* 
Technical and Commercial Courses in the Junior 
Secondary Schools 

The mean IQs and Qualifying scores for the junior secondary 
courses were as follows: 


Course 

Commercial 
Boys* technical 
Girls* technical 


Qualifying score 


IQ 


M 

a 

M 

0 

+ •681 sigma 

•59 sigma 

110-2 

10-3 

+ •078 

•60 

IOI -2 

10-3 

-051 

•60 

98-7 

9-5 

course is apparently the 

one which 

A ^ « A ■ 

attracts 


me Debt oi me junior scconu^iy it la ~ 

two sections, those who take French and those who do not; and 
Mr Robert Robertson finds that the average IQs of these sections 
were ii4'4 and 106-1. The commercial group in the junior 
secondary schools is therefore somewhat better in quality than the 
French-Literary group in the senior secondary schools; and that 
part of it which takes French is of the same quality as the French- 
Technical group. 
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QJJALIFYING SCORE 
(sigma) 

Fig. 58.—DUtribution of Qualifying icorca for courrcs in junior secondary schools. 


System of Retention 

The retention system is a departure from the ‘clean cut,* which 
provides for the pupil who, if promoted, would have to go to a 
backward class, but who would be fitted for a normal secondary 
course if his attainments were increased by a further period in 
the Qualifying class. Our data provide some information as to its 
practical operation. 

Below we give details of the duration of the period of retention 
before promotion for the retained pupils in our group: 




Number who were fit for 


Fit only 
for 



Senior 

Junior 

Day School 

Shorter 

Tot:>l 

Retained for 

Leaving 

Leaving 

Certificate 

coune 

backward 

Certificate 

Certificate 

(Lower) 

cla$3 


I year 6 months . 



3 

3 

# % 

3 

t year 

t 

6 

^3 

54 

43 

97 

6 months 

1 

3 

16 

80 

77 

‘57 

No post-primary 







career . . 


♦ « 

• » 

• « 

21 

21 

Total . 

t 

9 

44 

«37 

141 

278 
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The following were the courses taken by pupils after retention: 


Number who were fit for 


Senior 

Coune taken Leaving 

Certificate 

Junior 

Leaving 

Certificate 

Day School 
Certificate 
(Lower) 

Shorter 

coune 

Fit only 
for 

backward 

cla$i 

Total 

Senior secondary . 

I 

3 

6 

6 

• • 

6 

Junior secondary . 

I 

6 

36 

108 

1 

109 

Backward 

4 t 

• # 

• • 

23 

X19 

142 

No post-primary 
career ♦ 

• # 

# • 



21 

21 

Total 

1 

9 

42 

*37 

141 

278 


The second set of data shows that the system serves a useful 
purpose. If we assume that all the retained group would have 
gone to backward classes in a clean-cut system, we see that it has* 
saved about half of them from that fate. While many of these 
would no doubt have been transferred later from the backward 
group, the advantage of keeping them in a normal Qualifying group 
is considerable. 

IMPROVEMENT AFTER SIX MONTHS’ RETENTION 

We present fig. 59 partly to give an idea of the improvement in 
standing resulting f^rom six months* retention, and partly as an 
example of a case where we could not see why retention was 
thought necessary. This girl, who was first presented at ii years, 
falls above our pass-mark for admission to a senior secondary course. 
She is described as persevering and self-reliant, and the report on 
the home is ‘considerable parental interest.* 

After six months* retention she improved her Qualifying score 
from +'82 sigma to +1*20 sigma, and then proceeded to the 
commercial course at a junior secondary school. Thereafter, she 
transferred to a senior secondary school where she had to take the 
first-year course, thus losing one and a half years in all. 

Recommendations with regard to Retention 

A detailed study of the data suggests the following tentative 
conclusions regarding the system of retention: 

The system of retaining pupils for six months is valuable in 
certain cases. 

Retention for one year should be resorted to only 

ceptional circumstances, and retention for longer periods 
should be abandoned. 
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Another striking point is that the number who, after retention, 
are found to be fit only for backward classes is rather high. If 
these pupils could have been detected at the first presentation 
they could have been sent to the backward classes at once, thus 
relieving the Qualifying teachers of some dead-weight, and giving 
a sounder educational treatment to the pupils themselves. It is 
certainly not to their educational advantage to keep them at a 
hurdle which is too high for them, and run the risk of further 
sapping their self-confidence. This is an aspect of the selection 
problem which would repay further attention by Education 
Committees. Like many others, it raises the question of the 
desirability of a preliminary sifting of the material at an earlier 
stage in the primary course and of a development of special classes 
within the primary school. 

If, in consultation with their head teachers. Committees will fix 
their policy in regard to backward classes, the type of course to be 

should be sent to them, and so on, it 
wou e possible to fix a border-line as for senior secondary 
courses At the moment the attitudes of the head teachers to 
these classes vary so much that the problem is insoluble. 
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The Backward Group 

The distribution of Qualifying scores for this group, shown in 
fig. 57, would lead us to suspect that a number of pupils had been 
relegated to the category in error. This is confirmed by the 
judgments of the teachers who taught them in the backward 
classes: 

Judgment of teachers as to degree of fitness 
of pupils sent to backward classes 

Fit only for backward class ..... 283 

Fit for junior secondary course after retention . . 26 

Fit for junior secondary course at time of trans¬ 
ference to backward class . .... 7 

Total. 316 

The number of errors in placement, while considerable, was 
therefore not unduly large. 



CHAPTER XXI 


PERCENTAGE OF PUPILS FITTED FOR (a) A 
SENIOR SECONDARY EDUCATION, (/») A JUNIOR 
SECONDARY EDUCATION OF JUNIOR LEAVING 
CERTIFICATE LEVEL 

This old problem, whose obituary notices have so often appeared 
in the writings of educationists, has an administrative vitality 
which has preserved it amidst the constantly changing ideas of 
secondary education. But present nomenclature makes it difficult 
even to state precisely what the problem is; and any answer to it 
is destined soon to go out of date. All we can do is to give estimates 
for secondary courses of the types that are offered now and with the 
standards that now obtain. 


SENIOR SECONDARY EDUCATION 

Distinction between Potential and Actual Successes 

From our data we can estimate the percentage of the Qualifying 
group who would actually have been successful in a senior secondary 
course. The diflFerence between this and the percentage who 
actually took and succeeded in such a course is a measure of what 
may be regarded as waste of national talent. 

In assessing the waste, from the point of view of the individual 
pupil, we must bear in mind that the difference is mainly in length 
of course, and that the needs of many clever pupils may be best 
met in the junior secondary school. But from the statistical stand¬ 
point the measure is too lenient. 


A typical case 

In fig. 6o we give the profile of a boy with a Qualifying score of 
+ r8o sigma who went to a junior secondary course and was 
judged by his head master as one who would have been a certain 
failure if he had gone to a senior secondary school. 


2 II 
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The head master’s view that this boy would have been a fahure 
in a senior secondary course was based upon knowledge of his un¬ 
favourable home conditions, where there was a constant struggle 



with poverty and where he received no encouragement in his 
studies. In interpreting the profile it has to be remembered 
that the follow-up marks are relative to the mean of his junior 
secondary course. 


ANALYSIS OF 78 CASES WITH QUALIFYING SCORES OF 

-{-I*I SIGMA AND OVER 

In discussing individual pupils with head masters of the jumor 
secondary schools we frequently came upon instances of the above 
type. The child had the necessary ability and attainment but 
would have succumbed to the resultant drag of his environmenta 
conditions. Sometimes the source of the trouble lay 
poverty of the home, and the parental eagerness for the c^d to 
make his contribution to its meagre resources. In such conditions 
the child had no home encouragement in his further education, 
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often a positive antagonism, the offspring of sheer necessity. 
Frequently we were told that the child was a member of a large 
family living in a house affording no facilities for the home study 
demanded of the senior secondary pupil. Even when the home 
conditions were moderate or good, other strong forces of the 
environment seemed to be arrayed against the pupil; his friends 
were all leaving school at 14 to become van boys or message boys, 
with their evenings free and a little money in their pockets, and 
he wished to leave with them. The attraction of the senior 
secondary school was far too weak to overcome these opposing 
pulls. 

In the junior secondary courses there were 78 pupils with 
Qualifying scores of +ri sigma or over who would have been 
failures in the senior secondary school if they had gone there. Six 
out of 7 pupils in the senior secondary group with this Qualifying 
score are successes; so we may take it that these 78 pupils had 
the necessary ability. Their cases were made the subject of 
independent study, and the following reasons why they would not 
have been successful in the senior secondary school were found: 


Reason for failure N 

Unfavourable home conditions . . . 18 

Lack of interest: desire to take up work . 16 

Personal qualities : laziness, delinquency, etc. 8 

Ill-health. ...... 4 

No suitable course in senior secondary school 3 

No known reason, apart from the fact that the 
pupil’s performance in the junior secondary 
school was unsatisfactory ... 29 

Total ... 78 


The categories overlap; and when more than one cause was 
operative we placed the pupil in accordance with what the head 
teacher thought to be the main one. 

It is difficult to disentangle the complicated motives that weigh 
with these young adolescents. In 16 cases the pupil was not 
interested in school-work and wanted to take up employment. 
Which is the cause and which the effect? Is there a natural 
developmental tendency which impels certain pupils at this age 
to leave the artificial sheltered environment of the school and seek 
the real life of the workshop? Or is it merely that the pull of the 
environment, or the lack of reality and attraction in the present 
courses, creates such a tendency? The whole problem is one of 
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grave social importance and merits a more scientific investigation 
than we were able to make as a side line to an already extensive 
Inquiry. 

True Measure of Waste 

If we wish a complete estimate of the waste we must subtract the 
percentage who actually succeeded in a senior secondary school, not 
from the percentage who would actually have been successful if they 
had gone there, but from the percentage who have the ability and 
attainment necessary for success. Even thus we do not get the 
full measure of the waste. In the first place, we get no informa¬ 
tion as to the number of pupils who failed to reach the level of 
attainment for success in the senior secondary course because of 
adverse conditions during their primary course. In the second 
place, we have no way of measuring the increase in the number of 
potential successes which would result if the senior secondary 
courses had the degree of reality and attractiveness to which 
many modern educationists aspire. 


THE THREE ESTIMATES REQUIRED 

It will be conducive to clarity if, at this stage, we list the three 
estimates at which we shall aim. These are: 

(1) Percentage of the Qualifying group who had the ability 

and attainment necessary for success in a senior 
secondary course. 

(2) Percentage who would actually have been successful in a 

senior secondary course. 

(3) Percentage who actually were successful in a senior 

secondary course. 


(i) Percentage of Pupils who had the Ability and Attain¬ 
ment NECESSARY FOR SuCCESS IN A SeNIOR SECONDARY 

Course 

The second and third of the above percentages are based upon 
judgments by the secondary head master as to whether the pupil 
had more or less than a 50 per cent, chance of being successful. 
We have to base the first upon a statistically determined 50 per 
cent, chance of success; and since this is a different thing we shall 
first try to make the distinction clear. 
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MEANINGS OF *$0 PER CENT. CHANCE OF SUCCESS* 

When we asked a head master to decide whether a pupil had more 
than a 50 per cent, chance of success his decision was an absolute 
one in the sense that it took account only of two things: the pupil, 
with his individual abilities, attainments, interests and environ¬ 
mental conditions, and the standard of the Leaving Certificate 
examination. It had no reference to any particular entrance test 
or to the probable fate of other pupils with the same entrance score. 

When we stated that a senior secondary pupil with a score of 
+ 70 sigma had just a little more than a 50 per cent, chance of 
success, we used the phrase in a different sense. The judgment in 
this case was a statistical one, based upon the fact that half of the 
pupils in that group with an entrance score of + *695 sigma on that 
battery are successes and half are failures. It had reference to a 
particular entrance test and to other pupils with the same score. 
If the battery happened to be a poor one, the point at which the 
pupil would have an even chance would be different. 


LEVEL OF ABILITY AND ATTAINMENT NECESSARY FOR SUCCESS IN 
SENIOR SECONDARY COURSE GIVEN FAVOURABLE ENVIRONMENTAL 
AND OTHER CONDITIONS 

As usual we use as Qualifying scores the average on the complete 
battery IQ + Q + S+Ts. Any point on this scale, such as +*695 
sigma, represents a definite objective standard of ability and 
attainment. Now we have found that pupils in the senior 
secondary group who are at this level on this battery have just 
an even chance of success. We therefore assume that, it the 
extraneous conditions which affect success were as favourable for 
the whole Qualifying group as they are for the senior secondary 
group, then + *695 sigma would be the point at which half of the 
pupils in the whole group would be successes and half failures. 
We also assume that, in these circumstances, if the whole group 
had gone on to the senior secondary school the correlation between 
Qualifying scores and success would have been -8. 

THE BEST ESTIMATE OF PERCENTAGE OF SUCCESSES GIVEN 

CERTAIN CONDITIONS 

If, for a certain battery, the normal correlation between Qualify¬ 
ing score and success is +*8, and if a pupil with a score of +*695 
sigma has just an even chance of success, what is the best estimate 
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we can make of the percentage of pupils who would be successful 
in the secondary course? At first sight it would seem that all we 
have^ to do is to find the percentage of pupils who, in a normal 
distribution, lie above + -695 sigma. This, as a matter of fact, 
is 24*4 per cent. We are therefore tempted to reason that 24*4 
per cent, of pupils have more than an even chance of success, and 
that this could legitimately be taken as the percentage who have 
the ability and attainment which fits them for senior secondary 
education. 

We must, however, bear in mind that we are here using the 
phrase ‘even chance of success’ in the second, or statistical, sense. 
At + -695 sigma the pupil has an even chance as predicted by the 
battery which gives a correlation of +*8. With a poorer battery 
the correct pass-mark would be different, and we should get a 
different percentage. 

We suggest that a better estimate may be found as follows: 



SUCCESS IN SECONDARY SCHOOL 
Fic. 61.—Best estimate of percentage of successes. 



AB is the correct pass-mark of +‘695 sigma. The volume 
ARB is the percentage of pupils who, as predicted by the battery, 
have more than an even chance of success. To find the best 
estimate of the required percentage of successes we have to find the 
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line CD which, with this correlation, would separate successes 
from failures. This is easily done,' and we find that it lies at a 
distance of +-556 sigma above the mean. In a normal distribu¬ 
tion the volume CBD would therefore be 28-9 per cent.; and 
28*9 per cent, of our group of 3,021 is 873. 

Actually our distribution of Qualifying scores is not quite 
normal. The percentage of pupils with scores above +*695 
sigma is not 24*4 but 25*7. If we take this into account we find 
that the volume of CBD would be 30* i per cent., and the number 
of successes in our own group, 909. 

It may be of some general interest to mention that of the 776 
pupils who fell above +-695 sigma on our Qualifying scores, 339 
went to senior secondary schools, 434 to junior secondary schools, 
2 were retained, and i actually found his way into a special class 
for backward children. • 

Our eitimate^ of whose statistical weakness we are fully conscious, 
is that the percentage of the Qualifying group who had the ability 
and attainment necessary for success in a senior secondary course 
is 30-1. 


(2) Percentage who would actually have been Successful 

IN A Senior Secondary Course 


We have already explained how we found the numbers of pupils 
who, in the opinion of the secondary head teachers, would have 
been successful in courses leading to the Senior Leaving Certificate, 
Junior Leaving Certificate, etc., and the complete data are given 
in Table XLIV.* 

Into this Table, one of the most important in the Report, is 
condensed a large amount of information. All we require at the 
moment is the second column, from which we see that the number 
of those who would actually have been successful in a senior 
secondary course is 464. Of the 452 who embarked hopefully on 
such courses, 320 were graded as successes and 132 as failures, while 
142 successes Cdinc from the junior secondary group* 

Our estimate is that the percentage of the Qualifying group who 

would actually have been successful in a senior secondary course 
ts Iy4. 

The percentage is disappointingly small, but from our experience 


' Equation^ABi8^= +.695. Equation of the regression line of the 
These lines intersect at ( + *556, +*695), 


A;-arrays 
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of the careful and competent way in which the head teachers made 
their final gradings we are convinced that it has a high degree of 
reliability. 

TABLE XLIV 


Numbers of pupils who would have been successful in various 

levels of secondary education 




Number who would have been successful in 


Group 

N 

Senior 

Leaving 

Certificate 

Junior 

Leaving 

Certificate 

Day School 
Certificate 
(Lower) 

Shorter 

junior 

secondary 

course 

Fit only 
for special 
cla^ei for 
backwards 


(0 

w 

(3) 

(4) 

(5) 


Senior secondary 

4 Si 

320 

34 > 

430 

448 

1 

4 

Junior secondary 

>.975 

142 

615 

>.*47 

>.933 

4* 

Retained . 

278 

2 

10 

43 

138 

140 

Backward . 

316 


% • 


33 

283 

Total 


464 

966 

1,720 

2,552 

469 


(3) Percentage who actually were Successful in a 

Senior Secondary Course 

From column (2) of Table XLIV we find that of the whole 
group of 3,021 only 320 actually took and were successful in a 
senior secondary course. 

Our answer is that the percentage of the Qualifying group who 
actually were successful in a senior secondary course is lO'y. 


Percentage of Total Population who would actually 
BE Successful in a Senior Secondary Course 

The above results are given in the form in which they will be 
most useful to Scottish Education Committees. But considerable 
general interest attaches to the percentage of the total p^ulation 
who would be successful in a senior secondary course. 
quite the same thing as the percentage found in (2) above; for 
some of the Qualifying pupils are presented more than once. 
To get the percentage of the total population we require a 
group that is ‘pure,’ in the sense that every pupil comes into it 
once and not more than once. Such a group is that promote 
to the various post-primary courses in a particular year; and, 
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after making slight adjustments in respect of a small proportion 
(21 out of 3,021) who had no post-primary career, we find the 
result to be as follows: 

As conditions arc, 16-7 per cent, of the total population would be 
successful in a senior secondary course. 

In Chapter II we found some evidence indicating that the 
children in the City in which the Inquiry was made were lower in 
intelligence and scholastic attainment than pupils in certain Areas 
in England. Few Scotsmen will hesitate to deduce from this that, 
for Scotland as a whole, the above percentages might have to be 
somewhat increased. 

Collected Results for Senior Secondary Education 

We can now collect into a single Table the complicated answer 
to the deceptively simple question in the title of the chapter. 

TABLE XLV 


Fitness for senior secondary education 


Group 

N 

Per cent. 

Total Qualifying group .... 

3,021 

100 

Pupils who had the ability and attainment 
necessary for success in a senior secondary 
course . . ... 

909 

30-1 

Pupils who would actually have been success¬ 
ful in a senior secondary course 

464 

iS -4 

Pupils who actually were successful in a senior 
secondary course ..... 

320 

iO'6 

Pupils who took a senior secondary course and 
were failures ..... 

132 

4-4 


Secondary Education for all 

Table XLV does not solve this problem, but it gives some facts 
which clarify it. If by secondary education we mean the courses 
at present offered in our senior secondary schools, with their 
present content and standard of work demanded from the pupil, 
then we must conclude that secondary education should not be 
for all but only for, at most, 30-1 per cent, of the Qualifying pupils. 
Even if there should be a considerable extension in the provision 
of newer courses, with Art, Music, etc., as the main subjects, we 
doubt if the percentage could be much increased. 
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Educational Waste 

Yet while we cannot support the doctrine of secondary education 
for all, interpreting it in the narrow sense mentioned above, 
Table XLV shows that the benefits of such education could be 
much more widely spread, and that their present restriction 
results in an educational waste which should be attacked without 
delay. We fully realise that the 4*4 per cent, of pupils who took 
a senior secondary course but turned out to be failures must not be 
written off as an educational bad debt; also that pupils of good 
ability, who go to a junior secondary course, have a road through 
technical and commercial colleges to the commissioned ranks in 
industry and commerce. At the same time there is much to be 
said for the view that, in their own and in the wider national 
interest, they should have had the benefits of the richer and more 
liberal culture of the senior secondary school. 

With these reservations we may state our conclusions as follows: 

At present the number of pupils fit for a senior secondary 
course, who embark on such a course, is 10*6 per cent, of 
the Qualifying group. 

By proper use of the bursary schemes and of the best existing 
methods of selection and guidance we should be able to 
raise this to 15*4 per cent. 

To reach the final goal of 30*1 per cent, we require: 

(a) A far-reaching improvement in social and environ¬ 
mental conditions. 

{b) An increase in the variety and attractiveness of the 
senior secondary courses. 


Percentage of Qualifying Pupils for whom Provision 
SHOULD BE MADE IN SeNIOR SECONDARY ScHOOLS 

For what percentage of Qualifying pupils should an Education 
Committee make provision in its senior secondary schools? This 
is a difficult question, the answer to which depends upon. 

Which of the figures in Table XLV ( 909 > 4^4 $20) wt 

accept as the number of pupils fitted for a senior secondary 

education. 

The group from which we make the selection. 

The battery used in selection. 
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In the City in which the Inquiry was made there were 320 pupils 
who took a senior secondary education and were successful in it. 
As we have seen, there is no possible way of ensuring that these 
320 and no others wiU be admitted; and the number we admit 
will depend on whether we make the selection from a group oi 
candidates, all of whom are desirous of a senior secondary educa¬ 
tion, or from the complete Qualifying group. The former being the 
more probable alternativ , 

and give results which will probably hold for any district m 
Scotland. 

If the number of pupils who desire, and would be successful in, a 
senior secondary course is 320, the number of places to be provided 
is 335. Of these 335 pupils, 34 would prove to be failures; and 
in addition we should reject 19 pupils who would have been 
successful. 

Putting the above result in percentage form, we may say that 
if I0‘6 per cent, of the Qualifying pupils desire a secondary educa¬ 
tion and are fitted to profit by it, places should be provided in the 
senior secondary schools for ii*i per cent, of the Qualifying 
group. 

If by improvement of social and educational conditions it should 
ever be possible to raise the percentage to 3°'^ secondary places 
would be required for 31*2 per cent, of the Qualifying group. 


JUNIOR SECONDARY EDUCATION UP TO JUNIOR 

LEAVING CERTIFICATE LEVEL 

Of the three estimates mentioned on p. 214 the first two were 
obtained by the methods already described. In the percentage 
who actually were successful we included, in this calculation, only 
those who obtained the Junior Leaving Certificate. It was not 
possible to use the corresponding figure tvith the senior secondary 
group. If we had done so, the estimate of educational waste 
would have been even more alarming; for many of the 10*6 per 
cent, of potential successes would not complete the course. 

Percentage who actually obtained the Junior Leaving 

Certificate 

In the senior secondary schools pupils who are proceeding to 
the Senior Leaving Certificate do not sit for the Junior, so we give 
statistics only for the junior secondary group: 
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Course 



Boys* and girls’ 
technical 

Commercial 

All courses 

Passed 

Failed 

S 8 

133 

191 

^ ^ 
Not presented because 

9 

20 

29 

not up to standard 

Did not sit for other 

16 

10 

26 

reasons than above 

18 


18 

1,711 

Left before examination 

1,491 

• • 

220 

Total 

1,592 

383 

L 975 

Percentage of Certifi¬ 



cates obtained 

36 

347 

97 


To secure our comparative figures we have now to add to the 
191 junior secondary pupils who obtained the Junior Leaving 
Certificate those pupils in the senior secondary schools who may 
legitimately be regarded as having obtained it. We can calculate 
this in a reliable way. First of all we must exclude all who left 
too early; for the remainder we have definite judgments from the 
head teachers as to whether they would have passed the examina¬ 
tion if they had sat it. Reckoned in this way, the number of 
senior secondary passes to be added is 269. We may therefore take 
it that the number who successfully undertook a junior secondary 
course was 460, or 15-2 per cent, of the Qualifying group. 

Collected Results for Junior Secondary Education 

TABLE XLVI 


Fitness for junior secondary education up to level of 

Junior Leaving Certificate 


Group 

N 

Per cent. 

Total Qualifying group ..... 

3,021 

100 

Pupils who had the ability and attainment 
necessary for success in the Junior Leaving 
Certificate ...... 

985 

32-6 

Pupils who would actually have been successful in 
the Junior Leaving Certificate 

966 

32-0 

Pupils who actually were successful in the Junior 
Leaving Certificate . . . • • 

460 

15-2 

Pupils who took a Junior Leaving Certificate 
course and were failures .... 

55 

1-8 
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Comparison with Results for Senior Secondary 

Education 

When we compare Table XLVI with Table XLV perhaps the 
most striking point is that the ability level necessary for the Junior 
Leaving Certificate is nearly as high as that for the Senior. 

The next point is that nearly all the pupils with the necessary 
ability would have gained the Certificate if they had completed 
the course. This was far from being so in the Senior Leaving 
Certificate courses, partly, no doubt, because their length might 
discourage a pupil willing to tolerate school up to the age of 
15 but no longer, and partly, it may be, because the junior 
secondary courses are more attractive to a large proportion of 
pupils. 

It must, of course, be kept in mind that the 15*2 per cent, 
regarded as having successfully undertaken a Junior Leaving 
Certificate course includes the I0'6 per cent, who are successfully 
taking the Senior Leaving Certificate course. For this reason the 
educational waste in the junior secondary schools is best assessed 
from the statistics given for the junior secondary group on p. 222, 
taken in conjunction with the fact that 615 ' of this group would 
have been successful in the Junior Leaving Certificate if they had 
completed the course. Of these 615 only 191 actually did obtain 
the Certificate, showing that we have considerable leeway to make 
up in our preparation of non-commissioned officers for industry 
and commerce. 


DAY SCHOOL CERTIFICATE (LOWER) 

Since the Inquiry was started in 1935 important changes have 
been made in the organisation and nomenclature of post-primary 
courses and Certificates. In the main these do not affect our 
results, and we have, up to this point, used the new terms. For 
example, the Junior Leaving Certificate was previously called the 
Day School Certificate (Higher), but when the name was altered 
the Scottish Education Department intimated that the standard 
required would remain unchanged.® 

* See Table XLIV, p. 218. 

• Circular No. lao, 9th June 1939. 
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The Day School (Lower) was previously awarded to pupils who 
had satisfactorily completed a two-year course; but the 1939 Code 
provides that pupils who leave school without having satisfied the 
conditions for the award of a Junior or Senior Leaving Certificate 
shall be entitled to receive from the Education Authority a state¬ 
ment showing the particular stage of advancement reached at the 
date of leaving school. 

While our statistics are thus out of date so far as the Day School 
Certificate (Lower) is concerned, we give a summary of results 
which may have some historical interest, and will complete the 
picture of the conditions prior to 1939: 


Course 



Boys* and Girls* 
technical 

Commercial 

All courses 

Passed .... 

584 

268 

852 

Failed .... 

3 * 

• • 

31 

Not presented because not 



up to standard . 

323 

IS 

338 

Left before examination 

628 

61 

689 

Did not sit for other than 




above reasons 

26 

39 

65 

Total 

1,592 

383 

L 975 

Percentage of Certifi¬ 




cates awarded 

367 

70 0 

431 


Some pupils who would have obtained the Certificate were not 
presented because they were proceeding to the Day School 
Certificate (Higher). These form the greater part of those who 
‘did not sit for other than above reasons’; and even if we give 
the whole 65 the benefit of the doubt the picture is somewhat 

depressing. , 

According to Table XLIV, 1,247 junior secondary pupils would 

have obtained the Certificate if they had completed the course; at 

most only about 917 out of 1,975 could be regarded as havmg 

reached this modest standard. 

Our border-line for success in the Day School Certificate 
(Lower) was -*145 sigma. 
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Percentages fit for other Forms of Post-Primary 

Education 

These comprise courses of a lower level than the Day School 
Certificate (Lower), and also special classes for backward pupils. 
As far as the latter are concerned we have little confidence in our 
results, for the head teachers had very varied attitudes to the 
backward classes. With this reservation we may say that about 
84'5 per cent, of the Qualifying group appear to be fitted for 
normal classes and 15-5 per cent, for classes for backward pupils. 
The border-line separating the two groups was at - 1-085 sigma. 


8 



CHAPTER XXII 


PROPORTION OF SENIOR SECONDARY PUPILS WHO 
SHOULD TAKE A TWO-LANGUAGE COURSE 

What proportion of senior secondary pupils should take a two- 
language course? This is a question the answer to which is 
relevant only to the present conditions in certain Areas in Scotland. 
A French-Technical course could be made as stiff as a French-Latin 
one, but this does not appear to be the general rule in this country. 
A common arrangement is that pupils taking only one language 
give extra time to some other subject already in the curriculum, 
thus lightening the pressure. Frequently a change to this lighter 
course is made only after a pupil has spent a year at the second 
language; and it was to devise a procedure to avoid this wasted 
effort that we studied the problem. 


Decisions as to Fitness for a Two-Language Course 

We dealt only with the senior secondary group, and for each 
individual pupil we asked the head teacher, at the end of three 
years, to give a decision on the following issue. If, knowing what 
you now know of this pupil, you had to enrol him in the first year, 
would you put him into a two-language or a one-language course? 
It was a difficult question, and we should not have been surprised 
if the head teachers had hesitated to attempt an answer; yet when 
we considered the individual pupils the difficulty was less than we 
had expected. There was little difference in the standards used 
by the different head teachers, for the border-line was approxi¬ 
mately the same in each of the five schools. 


NUMBER JUDGED FIT FOR A TWO-LANGUAGE COURSE 

The number of senior secondary pupils who entered upon, 
and the number who were fit for, a two-language course are 

as follows: 


226 



TWO-LANGUAGE COURSES 227 

Group N Per cent. 

Total number enrolled in senior secondary course 452 100 

Number who took a two-language course . . 27S 61-5 

Number who gave up the second langu.ige . . 35 77 

Number fit for a two-language course . . 165 36-5 


Of the 278 who embarked on the two-language course only 165 
were in the end judged to be fit. Even after the withdrawals the 
number taking the heavier course is still far too large. 

Selection for Two-Lancuace Courses 

The wastage of eflport on the part of teachers and pupils suggested 
by these figures is so considerable that Education Committees and 
head masters may desire to take some steps to reduce it. 

The best way to do this consists in finding the pass-mark for 
senior secondary pupils on the Qualifying scores for the complete 
Qualifying group at which the pupil has just an even chance of 
success in a two-language course. This was done by the methods 
already explained and comes out to be 1-435 sigma, a standard 
which could be used by the secondary head master as a rough 
guide in his first classification of entrants. It should not, of 
course, be treated as a hard-and-fast line. Many pupils above this 
point, through lack of desire or of literary abilities and tastes, 
might be well advised to take only a single language. On the other 
hand certain pupils who are below this line might be admitted 
if they had special linguistic ability; but such admissions should 
be made only with some hesitation. In Chapter XXV \ve show 
how this border-line could be used in practice. 

Rough guiding figures are that about 7*56 per cent, of the 
complete Qualifying group, and 36*5 per cent, of the senior 
secondary applicants, have the ability and attainment necessary 
for success in a two-language course. 



CHAPTER XXIII 


LEVEL OF INTELLIGENCE NECESSARY FOR SUCCESS 
IN THE VARIOUS LEAVING CERTIFICATES 

SENIOR LEAVING CERTIFICATE 

These problems are mainly of theoretical interest, for we trust that 
no one who has studied fig. 42 ^ would use IQ alone as a basis of 
selection. We have also referred to the statistical pitfalls in trying 



inteilicence quotient 

Fio. 62.—Curvei ihowing the percentage of failure* In the Senior Leaving 

Certificate for difiercot IQi* 


to deduce, from such data as we give below, the percentage of pupils 
fit for secondary education. Certain results have, however, been 
pubUshed on the subject, 2 and we give our own for comparison. 

1 Fo/«ample, F. M. Earle, oj Ability for Secondary School Courses 

PubUcations of the Scottish CouncU for Research in Education No. X. London. 

University of London Press, Ltd,, 1936* P- 14* 

228 







229 


IQ FOR LEAVING CERTIFICATES 

The graphs in fig. 62 show how uncertain would be any pre¬ 
diction of success based on IQ alone. Pupils with very low IQs 
may pass; pupils with very high IQs may fail. The following, 
indeed, is the only answer which we can give to the question on 
the basis of the Inquiry results: 

Pupils with an IQ of 108 have an even chance of success. 

Pupils with IQs under 91 are almost certain to fail; those 
with an IQ over 130 are almost certain to pass. 

Two points must be kept in mind in interpreting these results: 

Our test of success was that the pupil would get a minimum 
Leaving Certificate in a suitable course in six years. 

Our IQs are for tests with a mean of lOO and a standard 
deviation of 15. 

Comparison with Results obtained by use of 

Otis Advanced Test 

For a number of years Miss Young collected results obtained by 
students in the application of the Otis Advanced Test to Leaving 
Certificate classes where the examination results were known. Her 
study of these, in which she had the assistance of Mr Hugh 
Stewart, B.Sc., and Mr David C. Johnstone, M.A., shows that 
a satisfactory solution of our problem cannot be found by this 
method unless the numbers are much larger than the 595 already 
obtained. The main difficulty, which is increased by the practice 
of not presenting all the pupils in the Leaving Certificate classes, 
is that the numbers with the lower IQs are too small, and that 
the group is too highly selected. The numbers who passed and 
the numbers who failed in the Senior Leaving Certificate for the 
various IQs are given on p. 230. 

The data have been partly condensed, but they show that two 
pupils with IQs of 126 have failed and that one pupil with an IQ of 
95 has passed. These actual results of presentation at the examina¬ 
tion may carry greater conviction than those of our Inquiry where 
the success gradings were prophetic. 

The data also bring out the statistical difficulties in determining 
from them the point at which the child has an even chance of 
success. A fair guess may, however, be made by smoothing the 
curve of percentage of failures at the parts where it is reliable and 
completing it in accordance tvith what seems to be its general 
tendency. This is the broken line in fig. 62. 
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IQ from Otis 

Number who 

Number who 

Advanced Test 

passed 

faUed 

135 

I 

• « 

130-134 

12 

« • 

127-129 

P 

• • 

126 

18 

2 

125 

29 

I 

124 

39 

4 

121-123 

107 

II 

118-120 

88 

27 

115-117 

68 

26 

112-114 

38 

26 

109-111 

*9 

16 

106-108 

13 

4 

103-105 

4 

4 

100-102 

• • 

2 

97-99 

1 

3 

96 

» « 


95 

1 

« • 

Total 

The results are as follows: 

469 

126 


Pupils with an IQ of 109 have an even chance of success. 
Pupils with IQs under 92 are almost certain to fail: those 
>vith IQs over 128 are almost certain to pass. 

The conformity with our Inquiry results is therefore reasonably 
close. 


JUNIOR LEAVING CERTIFICATE 

The problem is of even less practical value for the Junior Leaving 
Certificate, and it did not warrant the labour of making special 
distributions. We know, however, that 615 out of the 1,975 
junior secondary pupils were successes, and we know the correla¬ 
tion between IQ and success for this group. We can therefore 
obtain an estimate of the border-line by the method described 
in Chapter XIX. It comes out to be an IQ of 109. 

The fact that this is higher than that just found for the Senior 
Certificate in the senior secondary group will occasion no surprise 
to the reader who has grasped the principles laid down in 
Chapter XIV and who bears in mind the nature of the groups and 
the proportions of the unfits. 
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Comparison with Results obtained by use of 

Otis Advanced Test 

Miss Young has collected results for 469 pupils where we know 
both the Otis IQ and the results of the Junior Leaving Certificate 
examination. These are given below, and while they do not 
provide any statistical basis for the answer to our question, they 
may act as a warning to anyone who thinks of misusing our results 
by advising a pupil with an IQ of less than 109 to abandon the idea 
of taking a Junior Leaving Certificate course: 


IQ from Otis 

Number who 

Number who 

Advanced Test 

passed 

failed 

130-U+ 

3 

• « 

125-129 

26 

• • 

120-124 

72 

2 

115-119 


7 

1lo-i14 

104 

23 

105-109 

61 

21 

100-104 

20 

2 

95-99 

S 

4 

90-94 

I 

• • 

85-89 

1 

• « 

Total 

410 

59 


DAY SCHOOL CERTIFICATE (LOWER) 

By using the method of Chapter XIX we find that in the junior 
secondary group a pupil with an IQ of 94 has an even chance of 
obtaining this Certificate. 



CHAPTER XXIV 


ATTAINMENT AND NECESSITY 


We have taken the title of this chapter from the admirable Report 
which the Glasgow Education Committee has published on its 
examination held in May 1939,^ and we offer the following 
results as a small additional contribution to the study of this 
important social problem. 

No one who has handled the data of an Inquiry like ours could 
fail to be struck by the varying grades of material with which the 
different Qualifying teachers have to work; nor could he be free 
from doubt as to whether full recognition is given to the splendid 
work being done by some of the teachers with classes of poorer 
ability. We give below a comparison of two schools with different 
qualities of Qualifying material: 


Social 

Average 






rating 

health 

Mean 


Mean 


of 

grading 

percentage 




school 

of pupils 

Qe 

Qa 

IQ 

EQ 

AQ2 

A 

3*47 

650 

68-2 

116-0 

114-3 

113-5 

D 

2-95 

32-2 

248 

896 

897 

902 


One feels that a comparison of this kind should be supplied^ to 
every newly appointed Inspector of Schools with the instruction 
that it should be prominently displayed on his desk when he is 
writing his reports. Indeed, it is difficult to believe that without 
the help of the scientific test results it would ever be possible to 
do justice to the work done by the Qualifying teacher in the 
second school. From the examination results we note that the 
average mark in English in the second school is less than half that 
in the first: in Arithmetic it is considerably less than half. The 
difficulty in realising that the teachers of the second school are 
neither lazy nor inefficient is due largely to the difficulty of believ- 


* Statistical Report of Examination held in May 1939 ' Corporation 0 

Glasgow Education Department. ^ , 

• We use the sjmibol AQ to denote Arithmetic Quotient, not Accompus 

ment Quotient. 
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ing that the mean IQs of two schools could differ by as much as 
26-4 points. Yet this is actually so; moreover, from the last two 
columns we see that the teachers in the second school are making 
more of their opportunities than those in the first. (If the teacher 
is making full use of her material the EQ and AQ should be the 
same as the IQ.) 

Social Ratings of the Schools 

For some years before the Inquiry started we had been studying 
the relationship between intelligence and necessity, the investiga¬ 
tions being conducted by Miss Young and Dr McIntosh, assisted 
by Mr James P. Shepherd, M.A. For this purpose the schools were 
graded into four classes as regards necessity by the School Medical 
Officer, Dr A. E. Kidd, who brought to the problem a very inti¬ 
mate knowledge of the schools and of the districts from which 
they draw their pupils. He decided upon the grouping, after much 
experiment, because he felt that there was a clear gap between 
each class, that there was reasonable homogeneity within each 
class, and that in going over the list at different times he 
allocated the schools to the same classes without hesitation. 
Confirmation was obtained by submitting the lists to several 
others who were specially conversant with the social conditions of 
the City. The grouping was revised at the time of each investiga¬ 
tion, and a few alterations made in the light of changing social 
conditions. 

Levels of Intelligence in the Different Groups of 

Schools 

Five surveys of the intelligence of the schools at intervals since 
1931 have been made under Miss Young’s direction. On the 
Northumberland II Test the mean IQs of the different categories 
range from 117 to 102, while on the Simplex Junior Test they range 
from 109 to 83. We may therefore assume that the IQs obtained 
by the different tests are not comparable; and to facilitate com¬ 
parison we transformed the data in such a way as to make the 
mean IQ 100 and the standard deviation 15 for all tests. The 
mean IQs for the different groups of schools are given on p. 234. 

These figures reveal a striking fall in level of intelligence as we 
descend the scale of necessity, and we hope that they may do 
something to ensure that full recognition is given to the teachers 
in the poorer Areas who have to struggle year after year with 

8 * 
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difficulties not always appreciated by Education Committees 
That the results are worthy of confidence is shown by the remark¬ 
able consistency from year to year: 



193^ 

1932 

1932 

*933 

1936 


Northumber- 

Northumber¬ 

Mental 

Simplex 

Junior 

(AU 

Senior III 
classes) 

MHT 21 

Social 

land I 

land 11 

Survey 

0une 

rating 

(All QuaUfy- 

(All Qualify¬ 

(Age-group 

Qualifying 


ing classes) 

ing classes) 


clastet) 

A 

iii<5 

111-9 

iii*7 

nx-8 

113-1 

B 

I02«$ 

104-6 

100-7 

103-0 

106-8 

C 

98.9 

98-s 

98.9 

99.8 

99-5 

D 

96*6 

96.4 

98-3 

95-8 

96-1 


Variation in Level of Intelligence in Individual 

Schools 

The averages given above may be steadied by weight of numbers, 
and it is interesting to find out whether the teacher in a D school 



has a steady succession of poor classes, or whether she may 
occasional good year. The answer is given in fig. 63, which shows 
the fluctuations from year to year for a good school in class , a 
school about the middle of class B, and a school low in class 
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The IQs have been made comparable according to the procedure 
described above. 

The variations are considerable and show that it would be 
unfair to judge the work of a Qualifying class even on the basis 
of a sound general grading of the school. In the D school the 
teacher of the Northumberland II group had a much better 
chance than her colleague who had the group taking AIHT 21. 
There appears, in fact, to be no escape from the conclusion that 
no published assessment of the work of a teacher should ever be 
based on a general impression of the intelligence of the class or of 
the normal intelligence level of the school; scientific determina¬ 
tions of the intelligence and attainment levels of the individual 
class are essential. If this principle be confirmed, a revision of the 
system of school inspection would appear to be desirable. 

School Attainment in the Different Groups of Schools 
The following data are for the June group only: 


Social Mean 

rating percentage 



Qe 

Qa 

IQ 

EQ 

AQ 

A 

7I-I 

58-9 

1131 

113-8 

iii-S 

B 

607 

5^-4 

1068 

io6-2 

108 0 

C 

468 

420 

995 

997 

lOI 0 

D 

41-4 

33-6 

961 

95-6 

97-1 


Both in English and Arithmetic we find the same steady fall as 
we descend the scale, and the drop from the highest class to the 
lowest is very considerable. Yet the quotients show that in all 
groups the teachers are making good use of their opportunities, 
particularly in regard to Arithmetic. 

Comparison of the Groups of Schools in regard to 
Health, Industry and Practical Ability 

Only for health could we regard the gradings as strictly com¬ 
parable. For the other criteria the differing standards used by 
the teachers constitute a difficulty; yet it is one which would 
tend to mask the differences rather than create them. In practical 
subjects the teacher of a poor class, having little experience of 
really good classes, might tend to give a higher proportion of A 
and B gradings than a uniform standard for the whole population 
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would justify. If, therefore, we find that there is evidence of a 
fall, it is, for this reason, all the more convincing. 

Miss Young made the following comparison by giving 5 points 
for an A, 4 for a B, and so on: 


Social rating 

Health 

Industry 

Practical subjects 

A 

3-38 

3*32 

3-17 

B 

3-17 

3-17 

3-23 

C 

3-17 

3-09 

3'o8 

D 

316 

2-98 

3-05 


Such results have social implications which cannot be discussed 
here; but we present them in the hope that they may be of use 
to workers in this important part of the educational field. They 
suggest that the children of the better-class schools are not only 
more intelligent and of higher school attainment than those in 
districts where necessity is greatest; they are also healthier, more 
industrious and even of higher practical ability. The only devia¬ 
tion is that curious one in the last column, which indicates that 
the pupils in the B schools are better in practical ability than those 
in the A schools. This may be more than an accident. The B 
school pupils consist to a considerable extent of children of skilled 
artisans; the A schools are largely attended by children of pro¬ 
fessional classes. Speculation as to whether the difference is due 
to interest caught in the home, or to an atrophy of practical 
ability in the classes who have not to use their hands, may, however, 
wait until sounder investigations confirm its existence. 



CHAPTER XXV 

FIXING PASS-MARKS IN PRACTICE 

We have shown the importance of the correct fixing of pass-marks 
when examinations or batteries are used for selection, and we have 
hitherto determined these in terms of the standard deviation of 
the complete Qualifying group. We shall now show how they 
could be used in administrative practice; but before doing so, we 
shall consider briefly some other methods of fixing pass-marks. 

Can we trust to the Primary Teachers to fix the 

Border-Lines? 

We can make a rough estimate of the primary teachers’ pass- 
mark for admission to senior secondary schools as follows. For 
each pupil they gave a forecast of success on a five-point scale in 
which A and B indicated clear success, D and E certain failure; 
to the C category were relegated the doubtful cases. If we take 
the sum of the A and B gradings and add half of the C gradings, 
we get an estimate of the number of pupils who would fall above 
the pass-mark if it were drawn by the primary teachers. On this 
basis we find that they would pass about 36 per cent, of the whole 
Qualifying group; while the lowest of our scientifically determined 
border-lines (that is, that which would admit those who have the 
ability necessary for success, not merely those who would actually 
succeed) would pass 257 per cent. The primary teachers’ pass- 
mark for success in the senior secondary course is consequently 
too low. 

That the answer to the question is a negative is confirmed 
when we study the forecasts of the individual schools. In one 
school the primary teacher thought that 29 out of a class of 40 
would be certain successes in a senior secondary course; the 
external examiner thought that the number was 3; and, after 
having the pupils for three years, the secondary head masters 
found that 3 was the correct number. 

»37 
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Can wz trust to an External Examiner to fix the 

Pass-Mark? 


As far as our examiner is concerned the answer is an affirmative. 
The percentage of the whole group which falls above his pass- 
mark was 25-4; for our scientifically determined border-line the 
percentage is 25-7. One examiner is, however, too small a sample 
to form a sound basis for generalisation, and we made the foUow- 
ing further test. From the examination scripts of the December 
group Miss Young made up sets for English and Arithmetic 
separately, in which there was a graded series at intervals of I per 
cent, from the best to the worst paper. These were submitted by 
Dr McIntosh to a number of other examiners who were asked to 
read them over and, accepting the marking system already used, 
to indicate the point at which they would fix the pass-mark for 
admission to senior secondary schools. The test which they were 
asked to apply was that a child should be admitted if he had more 
than a 50 per cent, chance of obtaining a minimum Senior Leaving 
Certificate in a suitable course in six years. The following were 
the pass-marks fixed: 

Examiner 


Pass-mark Percentage of pupils 
per cent. admitted 


1 

2 

3 

4 

5 

6 

7 

8 


75-8 

9'8 

72*0 

147 

71-0 

165 

68-0 

21 7 

67-0 

23-5 

654 

266 

644 

284 

55-8 

467 


Inquiry examiner 68-0 

Correct pass-mark 66’8 


217 

23-9 


our 


From these results we may conclude that the answer to 
question is again a negative. 

So far as the examination is concerned, considerable improve¬ 
ment could no doubt be effected if the examiner were kept m- 
formed of the later success or failure of the candidates. From the 
principles which we have laid down he could ascertain what the 
number of failures should be if his pass-mark were correct, and 
he could then keep a set of standard answer papers. Yet question 
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papers vary in difficulty from year to year, and aUowance for this 

variation is far from easy. . r i_ i 

Even if we could rely upon the examiner to fix the pass-mark, 

there is the further complication that few Education Committees 

will now, we hope, use a single examination. The. problem of 

the future is therefore that of fixing the pass-mark for a battery. 

Possibility of admitting a Fixed Percentage of 
Candidates to Senior Secondary Courses 

In the year of our Inquiry 75 per cent, of the candidates for 
admission to senior secondary courses would have been admitted 
if the correct pass-mark had been used. With this percentage 
fixed for the year 1935-1936 it might be thought that we could 
apply it also in the year 1936-1937. We believe that this very 
simple system would work out just as fairly as most of the methods 
of fixing pass-marks which are ordinarily used; but it is unsatis¬ 
factory for various cogent reasons. It is fixed on small numbers; 
the quality of a small group of this kind may vary considerably 
from year to year; and the percentage for one Area might be very 
different from that for other Areas. 

The Method Recommended 

To secure an objective standard applicable throughout Scotland 
is a difficult problem, which may in the end be found insoluble. 
We believe, however, that the following method would give a fair 
preliminary approximation, and would be well worth a trial. It 
is based on the belief that the standard must be fixed in relation to 
a group so large and representative that it will vary little in quality 
and scatter from year to year. In City systems the complete 
Qualifying group would satisfy this condition; and we may 
further assume that the quality of these large groups would not 
vary greatly from Area to Area. If the variations were consider¬ 
able the Director of Education could determine a small allowance 
in the light of his experience. 

We have shown that in such a group the correct pass-mark 
for the best battery for candidates desiring admission to senior 
secondary schools is +*70 sigma. Now +*70 times the standard 
deviation of the whole Qualifying group represents a definite 
standard of intelligence and scholastic attainment, which will be 
steady from year to year and from Area to Area. We therefore 
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suggest that the soundest method is to test the whole Qualifying 
group and fix the pass-mark at of the standard deviation above 
the mean. We have given other reasons for the desirability of 
testing the complete group on p. 190. 


Practical Procedures 

In this section we show how the above method would work out 
in practice and how much clerical labour would be involved. To 
meet varying tastes, we begin with simple and inexpensive systems 
and end with the somewhat elaborate one which, in our opinion, 
would give the best results. 


A SIMPLE SYSTEM WHICH AVOIDS THE EVILS OF EXTERNAL 

EXAMINATIONS 

This would use the battery I+Tsi (intelligence scores and 
teachers’ estimates scaled on these) and the practical procedure 
would be as follows: 

Scale the teachers’ estimates in English + Arithmetic on the 
intelligence scores. 

Find the mean and standard deviation of the totals of the 
intelligence scores and teachers’ scaled marks. 

The pass-marks will be the following distances above the 
mean: 

(<2) Candidates for admission to senior secondary schools, 
•66 times the standard deviation. 

(b) Bursary candidates, -93 times the standard deviation, 
(f) Candidates for admission to two-language courses, 
1*39 times the standard deviation. 

The Education Committee or the Director of Education need 
have no fear of the statistical labour involved: a clerkess in an 
Education Office will readily master the statistician’s quick and 
easy method of calculating means and standard deviations. 

Suppose that the mean of the total scores is 103 and the standard 
deviation 30. The pass-mark for admission to senior secondary 
schools is at once found to be 103 +(’66 X30), that is, 123. 
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A SIMPLE SYSTEM WHICH INCLUDES AN EXTERNAL EXAMINATION 

The battery used in this case would be I -i-Q +Ts and the steps 
are as follows: 

Scale the teachers’ marks on the examination marks, taking 
English and Arithmetic together. 

Find the mean and standard deviation of the totals of 
I + Q + Ts. 

The pass-marks will be the following distances above the 
mean: 

(4) Candidates for admission to senior secondary schools, 
•70 times the standard deviation. 

(i) Bursary candidates, *97 times the standard deviation, 
(r) Candidates for admission to two-language courses, 

I’43 times the standard deviation. 

It is now hardly necessary to explain that this method allows for 
differences in the standard of difficulty of the examination papers 
from year to year. If the paper has been easy the pass-mark as 
calculated above will be higher. 

THE SYSTEM WHICH WE REGARD AS THE BEST 

This would be based on the battery I+Q + Ts, and would 
involve the following steps: 

Scale Te on Qe, giving Tes. 

Scale Ta on Qa, giving Tas. 

Find the mean and standard deviation of the totals of I, Qe, 
Qa, Tes and Tas. 

The pass-marks will then be determined as in the second 
system. 

An economical Committee could stop at that point. We 
should, however, like to tempt all Committees to undertake 
the following further steps which, we can assure them, are not 
so laborious or time-consuming as they would seem: 

Transform I, Qe, Qa, Tes, Tas and the average into standard 
scores. 

Enter these on cards, as shown in fig. 64, the right-hand 
side of which would be designed to meet the tastes of the 
Committee. According to our view the spaces for ‘health,* 
‘personal qualities* and ‘home conditions* would, in the 
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great majority of cases, be left blank. Entries would be 
made by the primary teacher only for applicants for ad¬ 
mission to senior secondary courses, and only when he felt 
that the pupil should be referred to the Medical Officer 
or to the staff of the Child Guidance Clinic. The right- 
hand side could also be adapted for follow-up records if 
that should be desired. 



Fig. 64.—Qualifying record card. 


No system which uses ordinary percentages could ever compete 
with standard scores which have a clear, defimte and constan 

significance. The lower pass-mark, shown by a 

Kne, is that for admission to senior secondary schools. 1 he otners, 

which also relate to senior secondary schools, are for bursaries an 

admission to two-language courses. should 

With this system the foUowing supplementary pnnciples sliouia 

be used in border-line cases: 

Refuse admission to aU candidates who fall 

the whole Qualifying group in any one of I, (Qe +Tes) an 

(Qa +Tas). 
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Admit pupils with average scores of -6 to -7 sigma if they are 
above 1-3 sigma in I, (Qe+Tes) or (Qa+Tas), and are 
not already failed under the first supplementary principle. 

If a pupil’s Qe or Qa score is less than the corresponding 
teacher’s scaled estimate by more than 7 sigma, such a 
score should be omitted in calculating the average. 

In fig. 64 the pupil has made a flop score in Arithmetic and 
would be an admit. He would not, however, be awarded a 
bursary; nor would he be admitted to a two-language course. 

Such cards, which can be interpreted at a glance, would be of 
considerable value to the heads of the secondary schools in advising 
pupils and parents and in making their first classification. 


THE SYSTEM RECOMMENDED FOR RURAL AREAS 

In rural Areas we have recommended that the battery used 
should be I+Q. No scaling of teachers’ estimates is required, 
and the steps are as follows: 

Add the intelligence and examination scores and find the 
mean and standard deviation. 

The pass-marks will be the following distances above the 
mean: 

(<i) Candidates for admission to senior secondary schools, 
•66 times the standard deviation. 

(b) Bursary candidates, -93 times the standard deviation, 
(r) Candidates for admission to two-language courses, 
1*39 times the standard deviation. 

It should be borne in mind that the pass-marks suggested above 
are based on data for a single industrial City. In aU Areas, but 
particularly in rural Areas, small adaptations may have to be made 
in the light of experience and of knowledge of the social conditions. 




APPENDIX A 

QUALIFYING EXAMINATION 


DICTATION AND SPELLING 

Tuesday, 26th November 1935 
9.30 a.m. to 10.15 a.m. 

DICTATION 

Instructions 

The passage should be read over once before \%Titing begins. 

The passage should then be dictated according to the phrases indicated. 

Phrases should be dictated twice, and all punctuation should be 
indicated. 

After the dictation is completed the passage should be again read over. 

These complaints | now became serious, j The admiral restrained 
them I by the calmness | of his countenance. | He called upon 
Heaven | to decide between himself | and the sailors. | He showed no 
fear. | He offered his life ( as a pledge, | if they would but trust 
him I and wait | for three days more. | He swore that, | if, | in the 
course of the third day, j land was not visible | on the horizon, [ he 
would yield | to their wishes | and steer for Europe, j The mutinous 
men | reluctantly consented | and allowed him | three days of grace. | 

SPELLING 

Instructions 

Each sentence should be read once and the class instructed to write 
down only the word underlined. The word may be repeated if necessary. 

1. Britain is a member of the League of Nations. 

2. The leader of the army was a very courageous man. 

3. The school had a half-holiday on Wednesday. 

4. On the top of the precipice there stood an old castle. 

5. A grocer must be able to distinguish between different kinds of tea. 

6. The mysterious disappearance of the visitor was the talk of the 

neighbourhood. 

7. Most new houses are lit by electricity. 

8. Please cut the cloth with a pair of scissors. 

9. The number of tramcars was not sufficient for the large crowd. 

10. When his arm was bandaged, the injured man felt relieved. 

*45 
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QUALIFYING EXAMINATION 


ENGLISH—COMPOSITION 

Tuesday, 26th November 1935 
10.45 a.m. 

(Time Allowed—35 Minutes) 


Write a composition on ONE of the following subjects: 

I. Tell the story of one of the most interesting incidents in the life of 
Bonnie Prince Charlie or Mary, Queen of Scots. 


2. Imagine you have moved into a new house. Describe the house, 

indicating its situation and surroundings, its appearance from the 
outside, the number of rooms and what each is used for, the 
garden, etc. 

3. You have spent a week^s holiday with your school friend. Write 

a letter of thanks and refer to some of your most pleasant 

experiences. 


4. The shops at Christmas time. 
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QUALIFYING EXAMINATION 


ENGLISH—GENERAL 

Tuesday, 26th November 1935 
1,30 p.m. 

(Time Allovs'ed—One Hour) 


Name 


School 


TURN OVER 
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TIME ALLOWED—ONE HOUR 

I. Read the following passage carefully and then answer the questions 
that follow. Write your answers in the spaces provided. 

In the course of long ages, the camel has become more and more 
suited to the country in which it lives, and to the work it has to do, 
so that it is quite indispensable in the deserts. The feet of the 
camel are broad and flat, and instead of being protected by hoofs, 
they have expanded cushion-like soles enclosed in a hard skin 
which is not sensitive to the burning sand. This kind of foot 
makes it possible for a camel to walk over sand bearing a heavy 
load or a rider, where a horse would sink and flounder at every step. 
The nostrils of the camel are slits which it can close at will to 
prevent the sand being blown into its breathing passages. Its hump 
is a storehouse of fat, and when it has to go for a time with very 
scanty food, the fat is gradually absorbed to nourish the body. 
When the camel reaches good fresh grass, or any kind of green 
plant, the hump quickly swells again. 

(a) Name three parts of the cameFs body which have become suited 
to its life in the desert. 


(b) Why is a horse not suited to the desert? 


(c) How does the camel manage to keep sand out of its breathing 
passages? 


(<?) What is the name for a fertile spot in the desert? 
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(/) When does it reappear? 


(g) What is the name for a storehouse for grain? 


(h) Write down the word opposite in meaning to each of the 
following: 

expanded . 

possible. 

gradually. 

(0 Write down the noun corresponding to each of the following 
verbs: 

protect. 

expand. 

enclose. 

(J) Write down the meaning of each of the following words: 

indispensable... 


sensitive 


absorbed 


TURN OVER 
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2. Read over the following sentences and then answer the questions 
below: 

(1) When the meal is over we sit down round the camp fire. 

(2) The servants begin to clear away what is left. 

(3) The guide we took with us is on guard. 

(a) Write down the subordinate clause in each of the three sentences 
and tell its kind and relation. 


(i) Tell what part of speech each of the six words underlined is. 



(f) Make six sentences in which each of these six words is used as a 
different part of speech; 


down 


round 


camp 


clear 


guide 


guard 
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3. Rewrite the following passage correctly, using where necessary 
commas, periods, question marks, quotation marks, capitals, etc.: 

John said tom have you prepared your lesson yes replied 
john as well as I can tom. 


4.. Combine each the following sets of short sentences into a single 
sentence with the same meaning, making any necessary changes. 
In doing so you may use any connecting word except “and.” 

(d) The bus arrived. The children were all very excited. It was 
taking them for a trip to the seaside. 


(i) 1 received a letter this morning from my uncle. He is a 
farmer. He invited me to spend my holidays with him. 
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QUALIFYING EXAMINATION 


ARITHMETIC—MENTAL 

Wednesday, 27th November 1935 

9 a.m. 


(Time Allowed—lo Minutes) 


Name 


School. 


1. Do not turn over this paper until you are told to do so. 

2. All working is to be done mentally. Only the answers 

are to be written down. 
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arithmetic—MENTAL 


No marks wUl be aUowed for answers that have been altered. 


1. 34 + 109 + 28+397. 

2. Take 99 from 10,001. 

3. Multiply 964 by 25. 

4. Find the total of 6/8^ 3 / 9 » 4 / 2 ^. ^/V* 

5. How much is left out of 14/9 after spend¬ 

ing 3/24 anti 2/94? 

6. How many i 4 d. stamps can be bought for 

half a crown ? 


7. Find the cost of l gross of pencils at i 4 'I- 

per pencil. 

8. If a cwt. of coal costs 1/6 what amount of 

coal can I get for 30/- ? 

9. I leave home at 10.55 a.m., walking at 

3 miles per hour. How far have I 
gone at 1.35 p.m.? 


Space for Answers 




3 - 

4 - 

5 - 

6 . 

7 - 




10. To-day is 27th November. On what day 
of the week did 30^ October fall 


10 . 
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QUALIFYING EXAMINATION 


ARITHMETIC—WRITTEN 

Wednesday, 27th November 1935 

9.25 a.m. 

(Time Allowed—One Hour) 

Note. —All the necessary working should be shown. 


1. {a) Subtract los 8jd from 6s 4d. 
(b) Divide £26$ 5s 4jd by 57. 

2. {a) Add together 7-324, 23-65, 101-008. 

(b) Multiply 36-289 by 1000. 


3. A train travelling at 45 miles per hour covers the distance between 
two telegraph poles in six seconds. How many yards are the poles 
apart? 


4. The following price list was hanging in a shop window: 

Flour, 2/8d per stone. 

Soap, 4d per bar. 

Custard Powder, 8d per packet. 


Sugar, 2^d per lb. 
Butter, i/id per lb. 
Eggs, i/2d per dozen. 
Tea, 2/6d per lb. 
Paraffin, l/zd per gallon. 
Candles, id each. 

Ham, i/iod per lb. 


Vinegar, 6d per pint. 
Pepper, per oz. 

Fruit Salad, l/3d per tin. 
Rice, 4^d per lb. 


Find out from the above price list the total cost of the following : 

2J stones Sugar. 

8 bars Soap. 

3^ lbs. Tea. 

I gallon Vinegar. 

I I gallons Paraffin, 
ij lbs. Pepper. 

8 packets Custard Powder. 

4 tins Fruit Salad. 

5. A silver collection was taken at a meeting. There were 15 half- 
crowns, 37 shillings, 8i sixpences and 150 threepences. The sum 
of 2 guineas was paid out of the collection for the rent of the hall. 
How much was left? 
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appendix B 

rating card 


(1) Name. 

(Block CapiuU, Surname Pint) 

(2) School. 

(3) Genxeal Inteluoince: I.Q. 


Date of Bieth 


Class. Section 


(4) General In- Humanistic Subjects RrALimc Subjects Practical Subjects 
Health dustry (Languag«> Eog- Hist.) (Anth., Nat. Study) 

Interest Attainment Interest Attainment Interest Attainment 



(5) Special talent in (i) Art, (u) Music, (iii) Games, Physical Exeecises, etc. 


(6) Physical defects which would affect choice of post-primary course: 


(7) Marked personal qualities 


(g) Home conditions which might affect choice of post-primary course: 


(9) PROBABLE SUCCESS IN 


Full Leatino 
CEE nncATE Course 

A 

B 

G 

D 

E 


vYeae Technical 3 -Year Literary 3-^*^ Commercial 
Course Cour« Course 
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APPENDIX C 

FOLLOW-UP CARD 

secondary schools 

1 . Name. 

(Block Capitals, Surname First) 

II. Date of Birth. 

III. School. 

IV. Date of Entering Secondary Dept. 

V. Course. 

VI. Final Date of Leaving School. 

VII. If the pupil is still on the roll at the end of session 1938-1939: 

(a) Has pupil completed a 3-year course?. 

(b) If SO, indicate by means of an X the one of the following cate¬ 

gories into which the pupil comes: 

(1) Passed the Day School Cert. (Higher). 

(2) Presented for Day School Cert. (Higher) 

but failed. 

(3) Presented for Day School Cert. (Higher) 
but result not known, e.g., because 

returning to school. 

(4) Did not sit for Day School Cert. (Higher) 

because not up to necessary standard. 

(5) Did not sit for Day School Cert. (Higher) 

because proposes to continue Leav. Cert. Course. 
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(r) Has pupil failed to complete a 3-year course?... 
(d) If SO, 

(1) W'^as this due to retardation in first year? . 

(2) Was this due to retardation in second year? 


VIII. 

Level of Success in 


( 7 ) 

(8) 

'15 Swi-P 

(0 

Englith 

(^) 

matict 

(3) 

Science 

(4) 

French 

(s) 

Latin 

(6) 

Special 

Subjecu 

of 

Coune 

General Level < 
Success in Cour 
[Leav. Certif. 
Standard] 

General Level < 
SucceM in Cour 
[Day Sch. Ccrti 
(Higher) Standai 

A 

A 

A 

A 

A 

A 

A 

A 

B 

B 

B 

B 

B 

B 

B 

B 

C 

C 

C 

C 

C 

c 

C 

C 

D 

D 

D 

D 

D 

D 

D 

D 

£ 

E 

E 

E 

E 

E 

£ 

E 

1 

IX. Are you satisfied that the 

course 

actually taken was the one most 


suited to the pupil’s abilities and educational needs? 

X. If not. 


(a) Do you think the pupil was unfit for promotion to a post- 
primary course at the time he/she was promoted ?. 

(A) In which other Secondary or Advanced Division course would 
the pupil’s needs have been best met? 


9 
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XI. Was the pupil^s success in the course affected by general health or 
physical defects ? 


XII. Occupation and/or Higher Educational Institution which pupil 
has entered, with name and address of employer (if known). 


t 


XIII. Other notes. 
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Correlations, correction for differences 
in means, 59, 6r, 62 
correction for range, 20, 57, 65, 66, 
, 7 L 74 

differences between courses, 79 
first- and second-year class marks, 
171 

maximum, with success to be ex¬ 
pected at Qualifying stage, 172 
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Day School Certificate (Lower), 223 
grading for success, 109 
IQ necessary for success, 231 
number obtaining, 224 
numbers capable of, 218 
pass-mark for, 224 
results for various courses, 224 
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Qualifying examination papers, 3 > 
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tests, 3, 11 
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Environmental conditions, 97 » * 54 > 
177, 182, 184, 195, 212, 220, 232 
Error of prediction, 23, 124 
Examiner, paid external, 6 
classification of pupils, 13 
pass-mark fixed by, 238 


External examination, escape from, 45, 
169 

selection system avoiding, 240 

Failure. See also Admit-fails 

percentage among those admitted 
to senior secondary courses, 141 
reasons for, 176, 177, 178, 179, 184, 
212 

Fitness. See Chapter XXI, 211 
for secondary education, 48, 104, 
123, 129, 211 

for two-language courses, 201, 206, 
226, 240, 241, 243 

Flexibility of secondary system, 170, 
186, 201 

Floor effect, 190 
Flop, 102, 147, 160, 243 
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card, 7, 256 

Follow-up marks. See Success in 
secondary course 

Glasgow Education Committee, 232 
Gradings, 4,13, 27, 29, 30, 38, 90,104, 
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Halo effect, 172 
Health, allowance for, 148 

average scores for senior secondary, 
junior secondary and backward 
groups, 204 

effect on success, 99, 176, 185 
in schools of different social ratings, 

23 h 235 

Medical Officer’s grading, 4 ) 9 °) 99 ) 
148, 205 

Industry, 27 

allowance for, 149, *55 
average scores for senior secondary, 
junior secondary and backward 

groups, 204 ... 

in schools of different soaal ratings, 

235 

relation to success, 155 
Inspection of schools, i, 232, 235 
Intelligence quotient (IQ) 

average for different jumor second¬ 
ary courses, 206 
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Intelligence quotient (IQ) 

average for different senior secondary 
courses, 206 

average for senior secondary, junior 
secondary and backward groups, 

204 

comparison with intelligence score 

(I). 73 

correlations, 64, 73, 84, 163, 190 
for Day School Certificate (Lower), 

for Junior Leaving Certificate, 230 
for Senior Leaving Certificate, 229 
in schools of different social ratings, 

232 

rise in predictive value in later years 
of course, 77 

teachers' estimates scaled on, 45,169 
Intelligence tests, 3, 9 

correlations, 19, 64, 73, 84, 163, 190 
Interests, 29, 185, 204 

Johnstone, D. C., 229 
Junior Leaving Certificate 
grading for success, 109 
IQ necessary for success, 230 
misfits for complete battery with 
correct pass-mark, 119 
number obtaining, 221 
numbers capable of, 218, 222 
pass-mark for, 118 
results for various courses, 222 
Junior secondary courses 
correlations, 68, 163 
number entering, 7 
number fit for, 13, 218 
predictive value of measures and 
batteries, 72, 75, 163 
Junior secondary group 
average IQ, 204 
distribution, 118, 203 
fitness for various levels of education, 
218 

4 

health, 204 
industry, 204 
interests, 204 

IQs and Qualifying scores for various 
courses, 206 
Qualifying scores, 203 

Kidd, J. C., 39, 83 
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McIntosh, D. M., 14, 25, 26, 82, 190, 
233> 238 

Marking, standards of, 14, 16, 34, 49, 
54> 78 

Medical examination, 186, 242 
Mental Survey^ 1932, 8, 234 
Minimum, imposition of, in separate 
tests and examinations, 143, 160, 
242 

Misfits. See Chapters XIII, 103; 

XV, 143; XVIII, 175 

actual and theoretical, i >4, 117, • 19 
boys outnumber girls, 187 
bursary examinations, 191, 194 
composition, 106, 108, 111, 114, ] 18, 

*38 

correlations corresponding to actual 
numbers, 117 
curves, 132 

determination of minimum for test 
or battery, 110, 129 
effect of allowances, 14S 
effect of applying supplementary 
principles, 143 

effect of raising pass-mark above 
correct point, 133 

minimum to be expected at Qualify¬ 
ing stage, 172 

minimum when pass-mark is fixed so 
as to admit pupils with more than 
a 50 per cent, chance of success, 
* 3 * 

number and composition in Junior 
Leaving Certificate, 119 
numbers for tests and batteries, 114 
order of predictive value based on, 

'•5 

pass-mark giving minimum for each 
battery, 112 

range of entrance attainment, 107 
use of profiles to determine numbers, 

*03 

value of Qualifying examination 
tested by number of, 105 
Mitchell, M., 82, 2cxa 
Moray House T«/j, 3, 8, 234 

Northumberland Tests, 233 

Omnibus school, 202 

Otis Advanced Test of Intelligence, 22 
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Partial correlation methods, 25 
Pass-mark. See Chapters XIII, 103; 
XIV, 120; XIX, 188; XXV, 237 
correct, 108, no, 112, 129, 131,136, 
137, 195 » 227 , 240 
effect of raising and lowering, 108, 

130, 131, * 33 , *34 
external examiner’s, 238 
for high ability in one subject, 145, 
160 

for Junior Leaving Certificate, 118 
for two-language courses, 227 
in bursary examinations, 195, 240 
primary teachers’, 237 
recommendation, 239 
spread of candidates in bursary 
examination, 189 

Personal qualities, 152, 157, 182, 185 
Plan of Inquiry, 2 
Practical ability, 205, 235 
Predicting success of an individual 

pupil, 123 

Predictive value of batteries for senior 
and junior secondary courses, 75, 

*63 

Predictive value of marks in first year 
of secondary course, 171 
Predictive value of measures, fall in 
later years of secondary course, 
76 

final order, 163 

order based on correlations and on 
misfits, 115 

order for junior secondary schools, 

72, *63 

order for senior secondary schools, 
67, 163 

significance of differences in final 
order, x66 

variation in school courses, 79 
Primary teachers, border-line fixed by, 

237 

estimates, 4, 6, 23, 27, 29, 34 , * 49 , 
168 

forecasts of success, 87, 151, 237 
reports, 4, 186, 242 
Probationary period, 17* 

Profiles. See Chapters XII, 86; XIII, 

*03 

$ample9) 91, I 7 ^> * 93 > 

Promotion system, 6 


Qualifying data, calculation of average 
of tests and examinations, 91 
correlations, 19 

primary teachers’ estimates, 4, 6, 23, 

27, 29, 34 , * 49 , *68 
rating cards, 4, 255 
reliability coefficients, 14 
standard errors of measurement, 15 
Qualifying examinations, 3 

correlations with bursary examina¬ 
tion, 190 

intercorrelations, 18 
papers, 245 

scatter of candidates compared with 
bursary examination, 189 
teachers’ estimates scaled on results, 

40 . 

value in selection for senior second¬ 
ary education tested by misfits, 
105 

Qualifying group, 2. See Chapter 
XX, 197 

comparison of boys and girls, 197 
correlations, 19 

desirability of testing whole, 42, 
*90, 239 
survey, 197 

Qualifying record card, 242 
Qualifyingscore(IQ + Q + S + Ts), 197 
average for senior secondary, junior 
secondary, backward and retained 
groups, 202 

average for senior and junior second¬ 
ary courses, 198, 206 
distribution for junior secondary 
courses, 207 

distribution for Qualifying group, 

*97 

distribution for senior secondary, 
junior secondary, backward and 
retained groups, 203 
Qualifying stage, i 

Questionnaire on choice of course, I 79 > 
180, 198, 201 

Questionnaire on pupils’ interests, 30 
Range, correction for, in correlations, 

57 » 65 ) 66, 71, 74 
Rating cards, 4 > 27 j * 48 » 25$ 
Recommendations, fixing of pass- 

marks, 239 
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Recommendations, follow - up tech¬ 
nique, 62 
retention, 208 

selection for secondary education, 
160, 168, 186, 239 

supplementary principles of selec¬ 
tion, 160, 168 
teachers’ estimates, 47 
Record Booklets, 7 

Refuse-successes in bursary examina¬ 
tions, 126, 191 

Reject-successes, 130, 139, 140, 173. 
See Chapters XIII, 103; XV, 143 
border-line cases, 180 
Qualifying scores, 180 
Reliability coefficients. See Chapter 

III, 14 

Retained group, 6, 7, 203, 204, 
207 

Retention system and results, 207 
Robertson, A. S., 83, 132 
Robertson, J. I. W., 27 
Robertson, R., 156, 198, 204, 206 
Rodgers, H. C., 179, 198 
Rural areas, system of selection for, 
169, 243 


Scaling of teachers’ marks, 20, 23, 38, 
58, 190 

Scatter diagram, 120 
Scholarships, selection for, 125, 188 
School subjects 

pupils’ interests. See Interests 
pupils’ preferences, 31 
Scottish Education Department, i, 
200, 223, 224 

Secondary education for all, 92, 219 
Selection for scholarships. See Bursary 
examinations 
general principles, 120 
problems, i, 2, 123 
system avoiding external examina¬ 
tion, 45, 169, 240 
system for rural areas, 169, 243 
system including external examina¬ 
tion, 241 

system regarded as best, 241 
Senior Leaving CertiBcate, 48 
IQ necessary for success, 228 
numbers capable of, 207, 211, 218 


Senior secondary courses 

ability and attainment necessary for 
success, 215 

admission of fixed percentage, 239 
number entering, 7 
percentage fit for, 211 
percentage successful, 218 
potential and actual success, 211 
predictive value of batteries, 75, 163 
predictive value of measures, 67, 163 
provision of courses, 187, 202, 214, 
219, 220 

provision of places, 139, 220 
significance of different sizes of 
correlation in selection, 142 
value of Qualifying examination in 
selection tested by misfits, 105 
Senior secondary group 
average IQ, 204 
average Qualifying score, 204 
comparison of sections taking differ¬ 
ent courses, 205 

distribution of Qualifying scores, 203 
health, 204 
industry, 204 
interests, 204 

numbers fit for various levels of 
secondary education, 218 
proportion capable of two-language 
course, 226 

Sex differences, effect of extraneous 
factors, 187 
industry, 28 

junior secondary group, 198 
misfits, 187 
pupils’ interests, 33 
Qualifying group, 197 
senior secondary group, 198 
Shepherd, J. P., 233 
Sirttplex yunior Group Test of Intelli- 
gence, 233 

Social rating of schools, 233 
Special classes for backward pupils, 6, 
7, 13, 207, 208, 209, 211, 218, 22s 
Standard errors of measurement for 
tests and examinations, 15 
Standard scores, 87, 105, 242 
Standardisation of tests, 2, 8 
Stewart, H., 229 

Success in secondary course. See 
Chapters VI, 48; XIII, 103 
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Success in secondary course, best esti¬ 
mate of percentage, 215 
comparability of marks, 49, 88 
comparison with bursary awards, 193 
correlations for tests and batteries, 

*63 

definition, 48 

50 per cent, chance, 91, 131, 215 
final assessment, 87, 104, 109 
maximum correlation to be expected 
at Qualifying stage, 172 
predicting, for individual pupils, 123 
primary teachers’ forecasts, 87, 151, 

relation to industry, 155 
relation to personal qualities, 157 
relation to teachers’ forecasts, 156 
scaling of marks, 58 

Teachers’ estimates. See Chapter V, 

34 

attainment in primary school, 4, 6, 

34 

industry. See Industry 
personal qualities. See Personal 
qualities 

pupils’ interests. See Interests 
scaling of, 20, 23, 38, 190 


Tests. See also Predictive value of 
measures 
Arithmetic, 11 

comparison with results for English 
pupils, 9 

correlations, 19, 22 
English, 3, 11 
intelligence, 3, 9 
length, 3, 14, 163, 166 
unstandardised, 74 
Thomson, G. H., 3, 9, 161 
Transference 

between courses, 174, 201 
between schools, 174, 200 
Two-language courses, 182, 187, 201, 
202, 240, 241, 243. See Chapter 
XXII, 226 

Variability of performance, 16, 100 
Variance and co-variance within and 
among schools, 63 

Waste, 173, 211, 220, 223, 227 
Weighting of measures, 68, 163. See 
Chapter X, 81 
Wilkie, R., 49 

Young, M., 50, 82, 229, 231, 233, 236, 
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